Drupal Planet

I used to apply patches from drupal.org by goeing to the issue page where the patch has been posted, downloading the patch to my desktop (or wherever) and then applying it using git apply /path/to/file.patch. Today I figured out a better way.

Open your favorite terminal application <cough>iterm2</cough> and cd to the git repository to which you want to apply the patch. Then type this: curl http://drupal.org/files/file.patch | git apply.

I feel like this is one of those tips that everyone else has known about forever and I'm just now having that aha! moment but in even if that's true, pretty nifty, huh?

If you have ever tried EFQ Views and it didn't work you might've thought it does not worth filing an issue because it was abandoned. It is no longer. Please file an issue telling me how it didn't work. Thanks.

read more

The Four Kitchens blog is running a story on how they used QueryPath and the Migrate module to migrate over 10,000 pages of content, in many different languages, into Drupal. I love to hear stories about the creative ways developers use QueryPath to accomplish complex tasks. A huge thanks to Mark Theunissen for the detailed write-up.

In related news, the new QueryPath 3 engine is just about done, and will make monster imports like this much faster.

read more

Over the years, I've accumulated a large collection of e-books and digital music albums, not to mention family pictures. Information overload is not a philosophical point of view, it's a real problem that forces me to devote time, effort and money to maintain that collection.

That's probably why so many media organizers exist. Because I believe that all applications should be delivered from the Web, and because no ready-made Web media organizer struck me as fulfilling my needs, I started to write my own using Drupal 6, dubbed Mediatheque.

read more

Tags:

• drupal
• planet drupal

At Vejle Idrætshøjskole we invite the community to participate in our some of our lectures. Of course we list the lectures on our lecture webpage. The list is easily created using a custom content type and some views.

However, I also wanted to make a version which prints nicely. A pdf is still the best tool to make a layout you can depend on. So I created a layout for my lectures using TCPDF. After a while I grew tired of the layout, and created a new layout still using TCPDF. But in doing so I was thinking: maybe other people could use my layouts, or maybe one day, I will like the old layout again. Therefore I decided that the content administrator should be able to choose between the different layouts I have created.

Enter ctools plugins system

I tried creating the layout utilizing the ctools plugin system. First I needed to make my module aware of the plugins:

/**
 * Implements hook_ctools_plugin_type().
 *
 * Has plenty options. See ctools/help/plugins-creating.html
 */
function vih_lectures_pdf_ctools_plugin_type() {
  $plugins['layouts'] = array(
    'use hooks' => TRUE,
  );

  return $plugins;
}


/**
 * Implements hook_ctools_plugin_directory().
 */
function vih_lectures_pdf_ctools_plugin_directory($owner, $plugin) {
  if (($owner == 'vih_lectures_pdf') && ($plugin == 'layouts')) {
    return 'plugins/' . $plugin;
  }
} 

Now, the system knows, that I am utilizing the ctools plugin system. Then I put my different layouts into different folders using the same naming convention, so I could use it in my module function. The function can both output one single lecture (a custom content type), but also a collection of lectures (created by an entity reference field on another custom content type).

 

function vih_lectures_pdf_booklet($lecture) {
  require_once libraries_get_path('tcpdf') . '/tcpdf.php';
  global $base_url;
  $events = array();
  $layout = variable_get('vih_lectures_pdf_layout', 'portrait');
  ctools_include('Layout', 'vih_lectures_pdf', 'plugins/layouts/' . $layout);
  $class = 'VIH_Lectures_Pdf_' . ucfirst($layout);

  if ($lecture->type == 'vih_lectures') {
    foreach ($lecture->field_lectures['und'] as $key => $value) {
      $node = node_load($value['target_id']);
      if ($node) {
        $events[] = $node;
      }
    }
  } else {
    $events[] = $lecture;
  }

  $pdf = new $class($events);
  $pdf->setBaseUrl($base_url);
  $pdf->setHeading($lecture->title);
  $pdf->setAuthor(variable_get('site_name', "Vejle Idrætshøjskole"));
  if ($logo = file_load(variable_get('vih_lectures_pdf_logo', ''))) {
    $pdf->setLogo(drupal_realpath($logo->uri));
  }

  if ($lecture->type == 'vih_lectures') {
    $pdf->setSubTitle($lecture->field_subtitle['und'][0]['safe_value']);
    $pdf->setDescription($lecture->body['da'][0]['safe_value']);
    $file = drupal_realpath($lecture->field_picture['da'][0]['uri']);
    $pdf->addFrontpage($file);
    $pdf->addSecondPage($file);
  }

  $pdf->render();
  drupal_exit();
} 

 

You can basically see the rest in the module file, which is the glue in the system. There I also created a simple settings pages (which could be hugely improved to look like e.g. the apperance page, so one could see the screenshot of the layout).

You can see the entire source code for my PDFs for the lectures at github.

How do you create your different layouts when using pdf's without overriding your previous layout on each iteration?

Versions of this talk has been presented at:

• Drupal Camp @ Stanford 2012
• 12 Nov 2011 Stanford Tech Briefing
• BADCamp 2011 (video available)

What are modules?

Drupal is designed to be modular. Instead of always having every possible tool or feature in every site's code, you can just have those you're actually going to use.

read more

Tags:

• drupal
• planet drupal

I am constantly trying to make it easier to be an editor at Vejle Idrætshøjskole site. Now the time has come to make it easier to create photo galleries. Earlier I uploaded the photo galleries to Picasa Web and created a custom input filter, which could embed the photo galleries as a slideshow into the page and used PWI to embed a photo gallery.

However, when watching the editors doing that, I just knew that I had to change the approach.

Enter media gallery

I had my eye out for media gallery for quite some time, but was reluctant to use it seeing the issue queue. However, watching screencasts I decided I would go for it anyways. Maybe I could help solve some of the issues.

My first problem was that I use the Media 2.x-branch, and media gallery only works for the 1.x-branch at the moment. Luckily, I was not the only one needing media gallery to work with 2.x. I started creating some patches which made it possible to have media gallery installed, but it was still not possible to add images to the galleries.

So this is where open source is great. leschekfm took my patches and made it possible to add files to the media gallery.

How to make media gallery work for the media 2.x branch

Media gallery still does not have a 2.x-branch. But to have media gallery work anyways, I did the following:

 $ git clone --recursive --branch 7.x-1.x http://git.drupal.org/project/media_gallery.git

$ cd media_gallery

$ git apply http://drupal.org/files/fixed-media-adding-and-multedit-1244204-comment-42.patch 

I only needed one patch, because leschekfm put my three patches into his, which makes testing a lot easier.

I also needed to install the required multiform module.

 $ drush dl multiform

$ drush en multiform 

Then for ease of upload, I installed plupload to make it easier to upload all the files at once.

 $ drush dl plupload

$ drush en plupload

$ cd sites/all/libraries/

$ wget https://github.com/downloads/moxiecode/plupload/plupload_1_5_4.zip

$ unzip plupload_1_5_4.zip 

So that is basically all there is to having media gallery work with media 2.x-branch.

Now we are just hoping for the developers of media gallery, that they will open up the 2.x-branch of media gallery, so the community can start helping out.

We're excited to welcome Tatiana (tvn) as the Association's first Drupal.org project coordinator. Keeping inline with our objective to make Drupal.org an amazing place for developers, site builders and contributors to collaborate we have begun to make investments to support the community.

read more

Downtown LA Drupal member Chris Paul (@cpjeeves) presents on a complex web application he built that has a desktop feel. Using AJAX, CKEditor and a host of other modules, his web application has an application-like experiance for the user and a nearly seamless content creation process. By leveraging the power of ctools command style AJAX, AJAX submit handlers, and a few custom upgrades, users are able to create content, view that content in relative lists, and then even edit and refresh those lists to suit their needs.

This kind of in-place create-and-edit system promotes a more intuitive creation process that doesn't break your train of thought while you rapidly input and modify your content. Drupal has always been capable of pushing the envelope. Join us as Chris shows us how he did it for this client project.

This video was recorded at the Downtown LA Drupal Meetup at Droplabs on March 20, 2012: http://groups.drupal.org/node/217564

Video: http://blip.tv/ladrupal/episode/6126726  
Comments: http://groups.drupal.org/node/217564#comments  

Tags:

• Planet Drupal
• Drupal
• LA Drupal

The first ever Drupal Mapping office hours will be held Thursday, May 10th, 2012 at 1PM EST in the #drupal-geo IRC channel (if you don't know about IRC, check out the Drupal IRC docs). Brandonian has spearheaded this effort, and will be joined by myself, phayes, and other great community members working in the Drupal geospatial sector. Besides offering general help with mapping, handling geospatial data, and related topics in Drupal, we will be discussing the following:

• Organizing a documentation sprint around an upcoming Drupal event TBD.
• Meta discussion about office hours in general/planning for next meeting.
• The state of geospatial in Drupal, specifically for a presentation I am giving at the Twin Cities Drupal Camp and possible at DrupalCon Munich.
• Event on Groups.Drupal.org

Topics: mappingdrupalIRCdrupal-planet

The Internet in the 90’s - a much simpler place.

We’ve done several migrations for clients who need their old, legacy content imported into Drupal from a collection of static HTML files. In this post I’ll outline the procedure we use to migrate, and provide some solutions to common problems related to encoding, line endings and parsing HTML with QueryPath. Code snippets are provided inline, and complete source code is provided as a Github gist.

1. Setup a Migration source

Let’s work with a hypothetical example site, which has the following directory listing:

$ ls /mnt/html
about.html
news.html
h1001.html
h1002.html
...
h1133.html
h1133.html
contact.html

The migration we’ll setup will specifically target the hxxxx.html files, as it’s a pretty common requirement to migrate hundreds or thousands of semi-structured files like this.

The main workhorse is the extremely well-architected Migrate module. If you haven’t yet discovered the wonders of it’s elegant abstraction and flexibility, my suggestion is to watch the presentations on the project page to gain a firm understanding of how it works. I won’t go into the basics here, as I’m going to cover only tips related to importing static HTML.

To use the Migrate module, you need to start with defining your migration. This is done by extending the Migration base class, and providing a constructor that injects the required dependancies into the object. The dependencies are a migration source, a destination, field mappings, etc. The migration source is what we’re interested in right now.

To assemble the source object we’ll need to create two things, the first is a MigrateListFiles, which is an object that provides a list of filenames. These filenames are used as ids in the Migrate module. You just need to create the MigrateListFiles with some parameters that direct it to the files you want:

$regex = '/h[0-9]+.html/';
$list_files = new MigrateListFiles(array('/mnt/html'), '/mnt/html', $regex);

The first parameter is an array of directories. In our case, we only have one, but there could be multiple different source directories. Next is the base dir, which is the part of the directory structure that you’d like explicitly excluded when ids are created. Finally, the regular expression that is used to filter the files in the directory down to just the ones we care about. In our case, we filter looking for any .html that starts with the letter ‘h’ and some digits. This means we won’t be migrating the individual about.html, contact.html, etc. pages.

We then create an object that provides a method of turning an id (filename) into a migrate-able chunk of data, which in this case means doing a file_get_contents() and returning the file contents to you:

$item_file = new MigrateItemFile('/mnt/html');

We then create an array which explicitly states the fields we’re going to be providing, and finally create the actual migration source, passing it the two objects from above:

$fields = array('title' => t('Title'), 'body' => t('Body'));
$this->source = new MigrateSourceList($list_files, $item_file, $fields);

To re-iterate, the MigrateSource is going to be a MigrateSourceList, which is a basic source that provides the ability to iterate over a MigrateListFiles and get data from a MigrateItemFile.

If having all these objects seems unnecessarily complex, remember that the Migrate module is extremely flexible - there’s a lot of code reuse going on here, so the tradeoff is worth it.

The rest of the source setup is beyond the scope of this article, and the Github gist contains a full example.

2. Fixing HTML file encodings

Old static sites typically get modified over the years by multiple people using many different platforms and editors, which results in a line ending and character encoding disaster. Any broken characters normally don’t show in the browser, even when the web server headers and HTML <head> tags suggest an encoding that conflicts with the actual encoding. This is because browsers are really, really good at sorting out the mess, which is something that they probably should not be doing - developers should be correcting problems in the source. But anyway.

The first step then, would be to use a good text editor to browse through a handful of files one by one, checking them for inconsistency. You may find several are detected as ISO-8859-1 (Latin-1) but are actually Windows 1252 and thus your files contain broken characters where there should be angled quotes and other Windows niceties. Assuming this is the case, you can use a snippet like this to convert the file contents to UTF-8 before doing anything else with it:

$enc = mb_detect_encoding($html, 'UTF-8', TRUE);
if (!$enc) {
  $html = mb_convert_encoding($html, 'UTF-8', 'WINDOWS-1252');
}

It’s pretty simple - if the encoding is not UTF-8 (which is very reliably detected by PHP, unlike WINDOWS-1252), then assume it’s WINDOWS-1252 and convert. If some of your content is in ISO-8859-1 this shouldn’t cause a problem since WINDOWS-1252 is a superset of ISO-8859-1.

Modify the above snippet according to the encodings you find in your source data.

3. Fixing line endings

There’s Unix, Windows, Classic Mac… and I’ve even discovered a crazy hybrid CRCRLF which I assume was created by a buggy editor. Best to convert everything to Unix (LF). Depending on your source, one way to do that is to simply blast the CR characters away:

$html = str_replace(chr(13), '', $html);

It’s worth noting that getting rid of the CR characters is a necessary step if you want to use QueryPath, as described below.

4. Fixing code point HTML entities

The correct entities to use in HTML for symbols are the “named” entities - for example the ™ sign should be ™. However, browsers also accept ™ for ™, which is a reference to a code point in the extended ASCII table. This would be fine, except in the case that the document is UTF-8, because ™ is not a valid unicode code point. The browser just cheats and assumes you mean the ASCII code point for ™, even though the correct code point in UTF-8 for ™ is ™. PHP isn’t so lenient, and will reject invalid values. Try it yourself:

print html_entity_decode('™', ENT_COMPAT | ENT_HTML401, 'UTF-8');
™
print html_entity_decode('™', ENT_COMPAT | ENT_HTML401, 'UTF-8');
 // Nothing! 
print html_entity_decode('™', ENT_COMPAT | ENT_HTML401, 'UTF-8');
™

So, if your source content has HTML entity code points in the extended ASCII range 127-159, you’re going to have to manually substitute them for named entities before trying to parse them in QueryPath, as QueryPath uses PHP’s default encoding functions, and thus will fail to generate the characters you want.

Luckily, this isn’t too hard:

function convertEntities($html) {
  $entities = array(
    '–' => '–',
    '—' => '—',
    '˜' => '˜',
    '™' => '™',
    // ... get them all!
  );
  $html = str_replace(array_keys($entities), array_values($entities), $html);
  return $html;
}

Here is the full list of ASCII characters.

5. Parsing HTML with QueryPath

You’re probably going to want to process and transform the source in some way, and for that, QueryPath is an excellent choice. It uses PHP’s native DOM handling abilities to load HTML and execute jQuery-like chainable commands. In fact, it provides most of the jQuery functions for your usage in PHP.

Getting the HTML into a QueryPath object can be tricky. Depending on how the source HTML is setup, each individual file may have a complete page with headers and footers, or it could include the sidebar and other components using Apache’s Server-Side Includes (SSI) and just contain the main content area (a saner approach).

In the case where you just have the body content in each file, you’ll need to pad it with the complete structure before loading it into QueryPath. To do that, use a function like this:

function wrapHTML($body) {
  // We add surrounding <html> and <head> tags.
  $html = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">';
  $html .= '<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body>';
  $html .= $body;
  $html .= '</body></html>';
  return $html;
}

I found that I always have to add this surrounding HTML in exactly this way in order for QueryPath to correctly use UTF-8. We can now create our QueryPath object, combining all the above tips in the correct order:

$html = str_replace(chr(13), '', $html);
$enc = mb_detect_encoding($html, 'UTF-8', TRUE);
if (!$enc) {
  $html = mb_convert_encoding($html, 'UTF-8', 'WINDOWS-1252');
}
$html = wrapHTML($html);
$html = convertEntities($html);
$qp_options = array(
  'convert_to_encoding' => 'utf-8',
  'convert_from_encoding' => 'utf-8',
  'strip_low_ascii' => FALSE,
);
$qp = htmlqp($html, NULL, $qp_options);

You can now perform your transformations on the $qp object. For example, you may want to process all anchors, so that you can change the ‘href’ on internal links to point at a new path:

// For all anchor links.
$anchors = $qp->top('a');
foreach($anchors as $a) {
  $href = trim($a->attr('href'));
  $href = getNewHref($href);
  // Set the new href.
  $a->attr('href', $href);
}

What about stripping out all those pesky Dreamweaver HTML comments? Easy:

foreach ($this->qp->top()->xpath('//comment()')->get() as $comment) {
  $comment->parentNode->removeChild($comment);
}

To get your body content out again, use the innerHTML function:

$body = $qp->top('body')->innerHTML();

Conclusion

These are some fairly tough problems to diagnose the first time you run into them, especially when dealing with a large body of content, so I hope this post helps you to get your clients safely into a Unicode-based Drupal world.

On NodeOne we use Features a lot. Mostly for exporting configuration, and we have some problems with that. Some of the stuff that you want to do, is not in Features, sometimes we need to solve it in another way - mostly with code in the installation profile, but some other issues just keep hanging lose.

Sometimes when I export permissions for a user, a need to add the permissions manually, but with Features Pipexplosion I just need to choose the role, and all the permissions and the dependencies of those permissions is added to my Feature. This make my work much easier.

Features Pipexplosion has been out since january, but when I downloaded it, onkly eight other people had done so, if the statistics are correct.

Another great addition to Features is a patch that speeds up the drush fra, sometimes we do often (if you do not use drush and develop Drupal, you should really learn how to use drush, it makes everything easier). The patch makes drush do the revert on the Features that are overridden, not all the Features.
 

Featuresdrush

In the comments of a past post discussing best Drupal modules, it was recommended not to use Webform for contact forms. So for a project I am currently working on I decided to just use the Drupal 7 contact module that comes with core Drupal.

I have used this is the past and it works just fine. My requirements were simple, "provide a site-wide contact form that authenticated users can use to contact the site admins". On the surface it seems simple enough.

Post Topics:  Drupal Drupal Planet General Discussion

read more

We are excited about our first conference in Latin America to be held in São Paulo, Brazil from December 6-8, 2012. This is a vibrant, new and growing Drupal community, and conference planning is well underway and the venue will be announced soon. Speakers from all around the world will be welcome to submit their proposals, as soon as the call for papers opens in the coming weeks.

Conference co-leads, Fabiano Sant'Ana and Fernando Paredes Garcia, in connection with the Drupal Association’s DrupalCon team, are looking for a dynamic group of volunteers to help plan this first-time event and a design company to theme the conference. 

read more

If you're like me, you've probably read a dozen or two articles about PHP performance in your career. Many of them are quite good, but some are simply flat out wrong, or misinformed.

One of the old truisms that has been repeated for as long as I can recall is "don't use readfile() if you have big files, because it reads the whole file into memory and your server will explode." The usual advice is to manually stream a file, like so:

<?php
$fp = fopen('bigfile.tar', 'rb');
while (!feof($fp)) {
print fread($fp, 1024);
}
fclose($fp);
?>

There's just one problem with that age-old truism: It's not true.

read more

At Pantheon we do all our code management via git. This lets us track operations, correct mistakes and allows a large amount of flexibility for workflows. By tracking an upstream for updates we can maintain core patches for people while providing a robust automated update functionality.

One place we do run into trouble is when there's a mix of management techniques used. If people unpack a tarball from drupal.org and check it into git (or upload it over SFTP using On Server Development), they may break their site: they'll be losing the Pressflow core and its ability to read configuration (e.g. mysql connection data) from the server environment.

Luckily, fixing that can be as easy as restoring boostrap.inc, but the legacy of adding a tarball can have other after-effects. Notably, a tarball from drupal.org contains .info files that have automatic packaging script data appended to them, like so:

; Information added by drupal.org packaging script on 2012-05-02
version = "7.14"
project = "drupal"
datestamp = "1335997555"

Drupal's core status checking code prefers this .info file data to all other sources when trying to determine its version. Moreover, the git history doesn't have any record of this, so if you wind up with legacy packaging info in your repository because of a tarball being in the mix at some point, updating via git (as Pantheon does) will leave you with updated code, but a Drupal installation that is mis-reporting its own version based on out-dated packaging information.

This is annoying (and common) enough that we wrote a quick PHP script to help clean it up. If you want to remove this cruft release packaging text from your core files, you can run the following on an up-to-date git clone of your codebase:

You can snag the script (or suggest improvements!) from the GitHub gist.

Pantheon's code import tools don't suffer from any of these problems — we run a heuristic to determine the imported major/minor version, rebase on Pantheon's upstream, and then bring the site up to date. This situation still arises whenever core updates come out though, or as the result of some complex or work-around-y manual imports.

Hopefully this helps some of our users, or other Drupal developers looking to move to a purely git-based workflow!

Start:  2012-07-26 (All day) - 2012-07-27 (All day) America/Chicago Drupalcamp or Regional Summit

The first Midwest Drupal Developer Summit will be held Thursday, July 26 through Friday, July 27 this year in Madison, WI. The event is free with advanced registration and will be hosted on the University of Wisconsin-Madison campus.

The event will include:

• Sessions similar to DrupalCon core conversations.
• Code sprints on Drupal 8.
• Introduction to Drupal core development and mentoring for new Drupal core contributors.

The developer summit will also be followed by DrupalCamp WI at the same venue on Saturday, July 28.

We are also seeking sponsorship for both events. Planning for the events is underway, so watch for more information soon! Contact: [email protected].

• Core

The Scotch (Blocks & Layouts Everywhere) Initiative needs some people to start digging through: http://drupal.org/node/1547742

I'm essentially trying to put together a plan for how we can build a mechanism for supporting on page context handling for any component that renders output, or makes layout decisions based upon the value of various contexts. I've taken the approach of using a separate Symfony dependency injection container for many reasons outlined in the post. I'd just like to have some more thoughts on it at this point and see if we can't put together a pretty solid plan for what the ultimate approach will look like here.

Thanks

Eclipse

• Core

Kris Vanderwater talks about the Drupal 8 Initiative Blocks & Layouts Everywhere. Kris gives a brief overview of what this is and explains how he is working on the usability side, bringing together designers and javascript experts to ensure this will be a great experience for all users. Kris goes so far as to refer to it as "Panels in Core" and is looking for contributors, so if that strikes your fancy, take a listen to what Kris has to say.

This Drupal Voices was recorded at the 2012 DrupalCon in Denver.

Release Date: May 3, 2012 - 10:00am Album: Lullabot Podcast Length: 8:00 minutes (5.56 MB) Format: mono 44kHz 97Kbps (vbr)

Start:  2012-05-03 19:00 - 21:00 UTC Online meeting (eg. IRC meeting) Organizers:  JohnAlbin

Please join us for our next IRC meeting to discuss issues related to the Drupal 8 mobile initiative. The meeting will be held in #drupal-mobile on IRC at 19:00 UTC on Thursday, May 3. Using IRC is easy; Drupal.org provides a guide to using IRC.

Thursday, May 3:
San Francisco — 12pm
New York — 3pm
London — 8pm
Taipei — 3am (Friday)
Convert to your timezone

Agenda

As a reminder, the scope of the D8 Mobile Initiative includes:
- Mobile-friendly Drupal admin
- Responsive design issues
- Front-end performance
- Converting existing D8 themes to be responsive

The agenda will be driven by the questions and suggestions given by attendees.

Please post your proposed discussion topics in the comments.

• Drupal 8 Initiatives