ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Add the Migrate module to your toolbox

?
?
Pieter-Jan Drouillon
Pieter-Jan Drouillon

Developing a new website or upgrading an existing one: both are challenges. And in most cases there is already content that needs to be copied to the new website. Meet the Migrate module! A splendid tool that already has proven itself, it is even included in Drupal 8 core (http://www.acquia.com/blog/d8migrate). In this talk you can expect: A short introduction on the migration process - ETL. Overview of the Migrate module Features of the migrate module During this presentation, all covered topics will be illustrated with experiences from two projects. The first is the migration of content of a legacy FAQ site to a brand new Drupal site. The original website was a completely customized web application where students can submit questions. These questions were then answered by the didactical team. The new site had to offer the same features while the content was conserved. The second and smaller project is the upgrade of an existing Drupal 6 site to a newly designed website in Drupal 7. As there was not that much content, migration was easier. As the design of Drupal 8 is Object-Oriented, there will also be some attention to the nice design of the Migrate module which results in an easy plugin system.

1 of 49
Download to read offline
Add The Migrate Module To Your Toolbox Pieter-Jan Drouillon
DRUPALCAMP 2014 GHENT OVERVIEW
OVERVIEW Introduction ETL Migrate module Various topics Overview
DRUPALCAMP 2014 GHENT INTRODUCTION
USE CASE Frequently Asked Question Site Mid 90¡¯s Custom application on LAMP No longer maintainable
WHY ABOUT FEEDS? Good alternative For ¡°easier¡± migrations
WHO AM I ? IT Expert & Tutor
WHO AM I ? Supply Chain Optimization Data Scientist
DRUPALCAMP 2014 GHENT E T L
ETL PROCESS E xtract T ransform L oad
DRUPALCAMP 2014 GHENT MIGRATE MODULE
USE CASE Students ask questions Didactical team answers Search TOC
BROWSE
BROWSE
BROWSE
EXTRACT MigrateSource MigrateSourceSQL MigrateSourceJSON MigrateSource*
EXTRACT idnr %tle mnr email date 20050021 1.6/a: x = tan(x) s0171503 bartvanhove@skynet.be Sunday 2 October 2005 20091791 4.33 taylorveeltermen bij r0297790 Saturday meerdere veranderlijken 26 November 2011 20090042 Kurk-?©\ en s0215276 Wednesday drijfstangmechanisme 28 October 2009 8846 energie inhoud van een condensator. s0109696 Saturday 8 May 10059 tussen%jdse toets s0164158 Wednesday 27 October 2004 10019 Fout in Vraag 2 ZelTest s0162924 kenzotomita@hotmail.com Tuesday 5 October 2004 10017 probleem bij het openen s0105157 eli.boudreaux@student.kul van zelTest 1 en 2 euven.ac.be Monday 4 October 2004
EXTRACT year Q# R course C# star s Q A %me 2005 21 u0010678 analyse 1 ster0.gif Naar aanleiding van het zoeken naar a (het minimum van het bereik van f(x)) stuit ik op de vergelijking sin<BR>(x) -?©\ x*cos(x) = 0 en dus x = tan(x).<BR><B R>Hoe kan ik x = tan(x) oplossen naar x? Ik herinner me dat ik hier een jaar geleden met mijn leerkracht <BR>wiskunde uit het middelbaar niet uitgekomen ben.<BR><BR Op zicht kan je zeggen dat x=0 een oplossing is van x=tan(x). Om de andere oplossingen te vinden, moet je numerieke methodes gebruiken die eventueel in een later jaar op je programma staan. Nu vragen we aan Maple 10 (classic worksheet) om de berekeningen uit te voeren. Klik <a href="antwoo rdFiles/ 20050021_1_ 16.38 2011 58 u0010678 analyse 4 Beste,<BR><B R>Is het mogelijk dat er in de oplossing van vraag 4.33 een fout is geslopen? <BR>Ik kom als derde term van de taylorveelter m -?©\9/100 uit ipv -?©\9/1000.<BR> <BR>Alvast bedankt Ik vrees dat er in jouw oplossing een fout staat. Met behulp van de Maple-?©\ demo kan je dit vlug nakijken. Neem hoofdstuk 4, paragraaf 5, voorbeeld 4.11. Verander de gegeven impliciet gedefinieerde func%e en het beschouwde punt en laat Maple het werk 15.54 2009 29 u0043795 mech 4 We zagen dit voorbeeld in onze nota's. Ik begrijp niet hoe je bij de posi%ebepalin g van B aan 2lcos(theta) komt.<BR><B R>Alvast bedankt Het punt B kan enkel volgens de x-?©\ as bewegen, dus de driehoek OAB is gelijkbenig. Zie figuur hieronder.<p> Nele<p align=leT><im g src=/slideshow/add-the-migrate-module-to-your-toolbox/41309300/antwoord Ajeeldingen/ 20090042_1_ 4.jpg></p> 8.30 2003 460 u0032325 nat 1 ster0.gif Bij de voorbeelden in het boek en degene die ikzelf heb uitgewerkt blijkt telkens dat de energie inhoud van een condensator zeer laag is (in de orde van enkele joules per m3) zelf met een Bij een photoflash willen we niet een grote hoeveelheid energie opslaan, we willen over een heel korte %jdspanne een heel grote hoeveelheid lading door het lampje jagen. De bamerij wordt 13.26 2004 59 u0018182 mech 9 ster0.gif Ik weet niet wat er er bedoeld wordt met 'tussen%jdse toets m¡Ì?t toetsplaporm' en bijgevolg dus ook niet wat het verschil is met een Het is het tweede, zoals je ondertussen wellicht al doorhad.<p>L uc Labey 10.55 2004 19 u0018182 mech 1 ster0.gif De vector a heeT als groome 7 en ax, ay = 3. Bepaal az en de drie hoeken met de Sorry, er stond nog een foutje in de opgave. a<sub>x</ sub> = 2 en niet 3. Dit wordt 15.18 2004 17 u0018182 mech 9 ster0.gif de pc in de bibliotheek geeT, nadat je op Maak je geen zorgen. Dit komt nog in orde.<p>Luc 15.55
class QuestionMigration extends Migration { ! public function __construct() { // Always call the parent constructor first for basic setup parent::__construct(); $this->description = t('Migrate questions from the source database to drupal users'); //source definition $columns = array( ! 0 => array('idnr','Database id'), ! 1 => array('title','Question title'), ! 2 => array('mnr','mnr of student asking the question'), ! 3 => array('email','Email of the stydebte'), ! 4 => array('date','When was the question asked'), ! 5 => array('year','Year of time '), ! 6 => array('questionnr','Nr of the question'), ! 7 => array('responder','Person who responded to the question'), ! 8 => array('course','Course question belongs to'), ! 9 => array('chapter','Chapter questione belongs to'), ! 10 => array('stars','#stars (obsolete)'), ! 11 => array('question','The question itself'), ! 12 => array('answer','Answer to the question'), ! ); //Make sure file is UTF encoded with BOM and has MAC linefeeds $file_path = ! DRUPAL_ROOT.'/'.drupal_get_path('module', 'vtmigrate').'/vragen.csv'; $this->source = new MigrateSourceCSV($file_path, ! $columns, ! array( ! ! 'header_rows' => 1, ! ! 'delimiter' => ';', ! ) );
TARGET CONTENT TYPE
MAPPING - PART 1 ! //map table $this->map=new MigrateSQLMap($this->machineName, ! array( ! ! 'idnr' =>array( ! ! ! 'type' => 'int', ! ! ! //'length' => 11, ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The id of the question', ! ! ),! ! ), ! MigrateDestinationNode::getKeySchema() ); ! //destination $this->destination = new MigrateDestinationNode('question');
MAPPING - PART 2 //destination $this->destination = new MigrateDestinationNode('question'); $this->addFieldMapping('title','title'); $this->addFieldMapping('body','question') ! ->arguments( ! ! array( ! ! ! 'format' => 'full_html', ! ! ) ! ); $this->addFieldMapping('field_answer','answer') ! ->arguments( ! ! array( ! ! ! 'format' => 'full_html', ! ! ) ! ); $this->addFieldMapping('created','date'); $this->addFieldMapping('field_course_term','course');
MAPPING - PART 2 //skip these source fields $this->addUnmigratedSources( ! array( ! ! 'year', ! ! 'questionnr', ! ! 'stars', ! ! 'time', ! ! 'course', ! ! 'chapter', ! ! //'answer', ! ! 'responder', ! ! 'mnr', ! ) ); //skip these destination fields $this->addUnmigratedDestinations( ! array( ! ! 'revision_uid', ! ! 'log', ! ! 'comment', !! ! //¡­ ! ! 'language', ! ! 'path', ! ) );
MAPPING - OVERVIEW
READY TO MIGRATE
READY TO IMPORT - OPTIONS
DRUPALCAMP 2014 GHENT VARIOUS TOPICS
DEPENDENCIES
DEPENDENCIES //add dependency $this->dependencies = array('AllChapterInOne');
DRUSH Better/faster than UI Start import Reasons why you should drush
DRUSH drush mi question --limit=¡°100 items¡± drush mi --rollback drush ms question
PREPAREROW public function prepareRow($row){ ! //setting creationtime to correct timestamp ! $row->date = $this->convertDateToTimestamp($row->date,$row->time,$row); ! return TRUE; }
PREPARE function prepare($question,stdClass $row){ ! $question->body['und'][0]['value_format'] = 'filtered_html'; !$question->body['und'][0]['format'] = 'filtered_html'; ! !$question->field_answer['und'][0]['value_format'] = 'filtered_html'; !$question->field_answer['und'][0]['format'] = 'filtered_html'; }
MULTI-KEY MAP //map table $this->map=new MigrateSQLMap($this->machineName, ! array( ! ! 'chapterid' =>array( ! ! ! 'type' => 'int', ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The chapter ID', ! ! ), ! ! 'course' => array( ! ! ! 'type' => 'varchar', ! ! ! 'length' => 10, ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The course ID', ! ! ), ! ! 'year' => array( ! ! ! 'type' => 'int', ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The year of the course', ! ! ),! ! ), ! MigrateDestinationTerm::getKeySchema() );
USER IMPORT class UserMigration extends Migration { ! public function __construct() { // Always call the parent constructor first for basic setup parent::__construct(); $this->description = t('Migrate users from the source database to drupal users'); //map table $this->map=new MigrateSQLMap($this->machineName, ! array( ! ! 'unr' =>array( ! ! ! 'type' => 'varchar', ! ! ! 'length' => 8, ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The KU Leuven unumber', ! ! ),! ! ), ! MigrateDestinationUser::getKeySchema() ); //¡­
USER IMPORT ! //source definition $columns = array( ! 0 => array('name','Name of the user'), ! 1 => array('unr','Unumber of the user'), ! 2 => array('email','Email of the user'), ! 3 => array('course','The course the user is involved'), ! 4 => array('monitor','Is this user a monitor?'), ! 5 => array('id','Database id'), ! 6 => array('roles','The courses involved (will be mapped to roles)'), ); //Make sure file is UTF encoded with BOM and has MAC linefeeds $file_path = ! DRUPAL_ROOT.'/'.drupal_get_path('module', 'vtmigrate').'/personeel.csv'; //drupal_set_message(fopen($file_path,'r')); //drupal_set_message($file_path); $this->source = new MigrateSourceCSV($file_path, ! $columns, ! array( ! ! 'header_rows' => 1, ! ! 'delimiter' => ';', ! ) ); //destination $this->destination = new MigrateDestinationUser(); //¡­
USER IMPORT - MAPPING //mapping $this->addFieldMapping('name','unr'); $this->addFieldMapping('mail','email'); //$this->addFieldMapping('role_names','roles') //!! ->separator(','); //skip these source fields $this->addUnmigratedSources( ! array( ! ! 'name', ! ! 'course', ! ! 'monitor', ! ! 'id', ! ! 'roles', ! ) ); //skip these destination fields $this->addUnmigratedDestinations( ! array( ! ! 'is_new', ! ! //¡­ ! ! 'picture', ! ! 'init', ! ! 'path', ! ! 'pathauto', ! ) );
USER MAPPING - PREPARE public function prepare($user,stdClass $row){ ! //add all the role ids to the user account ! $role_names = explode(',',$row->roles); ! ! //check if role didactisch team bestaat ! $new_role = array('didactisch team'); ! foreach($new_role as $r){ ! ! if(!user_role_load_by_name($r)){ ! ! ! //create it ! ! ! $role = new stdClass(); ! ! ! $role->name = $r; ! ! ! user_role_save($role); ! ! } ! } ! $role = user_role_load_by_name('didactisch team'); ! //$user role is a mapping from role id to role name ! $user->roles[$role->rid] = $role->name; }
DATE //as suggested in https://drupal.org/node/1806296#comment-6589830 public function prepareRow($row) { ! //for some reason, 2 hours have to be added ! $row->startdate = strtotime("+2 hours",$row->startdate); ! $row->enddate = strtotime("+2 hours",$row->enddate); ! $row->datespan = drupal_json_encode(array( ! ! 'from' !=> $row->startdate, ! ! 'to' ! => $row->enddate ! )); ! //change linebreaks back to newlines ! $row->body = str_replace('<br/>',"n",$row->body); }
REFERENCE public function prepareRow($entity,stdClass $row) { ! //¡­ ! $loc_query = db_select('node','n') ! ! ->fields('n') ! ! ->condition('n.type','location') //MYSQL's like is case insensitive ! ! ->condition('n.title','%'.$key.'%','LIKE'); ! $locations = $loc_query->execute(); ! ! ! ! ! if(!empty($locations)){ ! ! foreach($locations as $loc){ ! ! //target_id is specific for entity reference ! ! // cf https://drupal.org/node/1845986 ! ! $entity->field_location['und'][0]['target_id'] = $loc->nid; ! } }
USER MIGRATION $this->destination = new MigrateDestinationUser( ! array( ! 'md5_passwords' => TRUE, ) );
FILES & IMAGES Link to files in answer Be creative Use Regex :)
FILES & IMAGES $this->addFieldMapping('field_attachments','files') ! ->separator('#'); $this->addFieldMapping('field_images','images') ! ->separator('#');
FILES & IMAGES $files = array(); $nr_matches = preg_match_all('/<a href="(S+)"/i',$row->answer,$matches,PREG_SET_ORDER); //drupal_set_message("#match: ".print_r($nr_matches,true),'warning'); if($nr_matches > 0){ drupal_set_message("#filematch: ".print_r($matches,true),'warning'); ! $base_url = 'https://mirw.kuleuven.be/vragentrommel/trommelroot/'; ! //all matches are stored in an array at $matches[0] foreach($matches as $match){ ! $files[] = $base_url .$match[1]; ! } } //add the URI of the files to $row->files and separate them by hashtag $row->files = implode('#',$files);
FILES & IMAGES $images = array(); $nr_matches = preg_match_all('/<img src=/slideshow/add-the-migrate-module-to-your-toolbox/41309300/([^s>]+)>/i& $matches,PREG_SET_ORDER); //drupal_set_message("#match: ".print_r($nr_matches,true),'warning'); if($nr_matches > 0){ drupal_set_message("#image match: ".print_r($matches,true),'warning'); ! $base_url = 'https://mirw.kuleuven.be/vragentrommel/trommelroot/'; ! //all matches are stored in an array at $matches[0] ! foreach($matches as $match){ ! $file_url = $base_url . $match[1]; ! ! $images[] = $file_url; ! } } //add the URI of the images to $row->images, separated with # $row->images = implode('#',$images);
DRUPALCAMP 2014 GHENT CONCLUSION
DRUPALCAMP 2014 GHENT RESOURCES
RESOURCES Migrate Documentation Migrate Example BTMash
Q & A @pjdrouillon

More Related Content

Add the Migrate module to your toolbox

  • 1. Add The Migrate Module To Your Toolbox Pieter-Jan Drouillon
  • 3. OVERVIEW Introduction ETL Migrate module Various topics Overview
  • 4. DRUPALCAMP 2014 GHENT INTRODUCTION
  • 5. USE CASE Frequently Asked Question Site Mid 90¡¯s Custom application on LAMP No longer maintainable
  • 6. WHY ABOUT FEEDS? Good alternative For ¡°easier¡± migrations
  • 7. WHO AM I ? IT Expert & Tutor
  • 8. WHO AM I ? Supply Chain Optimization Data Scientist
  • 10. ETL PROCESS E xtract T ransform L oad
  • 11. DRUPALCAMP 2014 GHENT MIGRATE MODULE
  • 12. USE CASE Students ask questions Didactical team answers Search TOC
  • 16. EXTRACT MigrateSource MigrateSourceSQL MigrateSourceJSON MigrateSource*
  • 17. EXTRACT idnr %tle mnr email date 20050021 1.6/a: x = tan(x) s0171503 bartvanhove@skynet.be Sunday 2 October 2005 20091791 4.33 taylorveeltermen bij r0297790 Saturday meerdere veranderlijken 26 November 2011 20090042 Kurk-?©\ en s0215276 Wednesday drijfstangmechanisme 28 October 2009 8846 energie inhoud van een condensator. s0109696 Saturday 8 May 10059 tussen%jdse toets s0164158 Wednesday 27 October 2004 10019 Fout in Vraag 2 ZelTest s0162924 kenzotomita@hotmail.com Tuesday 5 October 2004 10017 probleem bij het openen s0105157 eli.boudreaux@student.kul van zelTest 1 en 2 euven.ac.be Monday 4 October 2004
  • 18. EXTRACT year Q# R course C# star s Q A %me 2005 21 u0010678 analyse 1 ster0.gif Naar aanleiding van het zoeken naar a (het minimum van het bereik van f(x)) stuit ik op de vergelijking sin<BR>(x) -?©\ x*cos(x) = 0 en dus x = tan(x).<BR><B R>Hoe kan ik x = tan(x) oplossen naar x? Ik herinner me dat ik hier een jaar geleden met mijn leerkracht <BR>wiskunde uit het middelbaar niet uitgekomen ben.<BR><BR Op zicht kan je zeggen dat x=0 een oplossing is van x=tan(x). Om de andere oplossingen te vinden, moet je numerieke methodes gebruiken die eventueel in een later jaar op je programma staan. Nu vragen we aan Maple 10 (classic worksheet) om de berekeningen uit te voeren. Klik <a href="antwoo rdFiles/ 20050021_1_ 16.38 2011 58 u0010678 analyse 4 Beste,<BR><B R>Is het mogelijk dat er in de oplossing van vraag 4.33 een fout is geslopen? <BR>Ik kom als derde term van de taylorveelter m -?©\9/100 uit ipv -?©\9/1000.<BR> <BR>Alvast bedankt Ik vrees dat er in jouw oplossing een fout staat. Met behulp van de Maple-?©\ demo kan je dit vlug nakijken. Neem hoofdstuk 4, paragraaf 5, voorbeeld 4.11. Verander de gegeven impliciet gedefinieerde func%e en het beschouwde punt en laat Maple het werk 15.54 2009 29 u0043795 mech 4 We zagen dit voorbeeld in onze nota's. Ik begrijp niet hoe je bij de posi%ebepalin g van B aan 2lcos(theta) komt.<BR><B R>Alvast bedankt Het punt B kan enkel volgens de x-?©\ as bewegen, dus de driehoek OAB is gelijkbenig. Zie figuur hieronder.<p> Nele<p align=leT><im g src=/slideshow/add-the-migrate-module-to-your-toolbox/41309300/antwoord Ajeeldingen/ 20090042_1_ 4.jpg></p> 8.30 2003 460 u0032325 nat 1 ster0.gif Bij de voorbeelden in het boek en degene die ikzelf heb uitgewerkt blijkt telkens dat de energie inhoud van een condensator zeer laag is (in de orde van enkele joules per m3) zelf met een Bij een photoflash willen we niet een grote hoeveelheid energie opslaan, we willen over een heel korte %jdspanne een heel grote hoeveelheid lading door het lampje jagen. De bamerij wordt 13.26 2004 59 u0018182 mech 9 ster0.gif Ik weet niet wat er er bedoeld wordt met 'tussen%jdse toets m¡Ì?t toetsplaporm' en bijgevolg dus ook niet wat het verschil is met een Het is het tweede, zoals je ondertussen wellicht al doorhad.<p>L uc Labey 10.55 2004 19 u0018182 mech 1 ster0.gif De vector a heeT als groome 7 en ax, ay = 3. Bepaal az en de drie hoeken met de Sorry, er stond nog een foutje in de opgave. a<sub>x</ sub> = 2 en niet 3. Dit wordt 15.18 2004 17 u0018182 mech 9 ster0.gif de pc in de bibliotheek geeT, nadat je op Maak je geen zorgen. Dit komt nog in orde.<p>Luc 15.55
  • 19. class QuestionMigration extends Migration { ! public function __construct() { // Always call the parent constructor first for basic setup parent::__construct(); $this->description = t('Migrate questions from the source database to drupal users'); //source definition $columns = array( ! 0 => array('idnr','Database id'), ! 1 => array('title','Question title'), ! 2 => array('mnr','mnr of student asking the question'), ! 3 => array('email','Email of the stydebte'), ! 4 => array('date','When was the question asked'), ! 5 => array('year','Year of time '), ! 6 => array('questionnr','Nr of the question'), ! 7 => array('responder','Person who responded to the question'), ! 8 => array('course','Course question belongs to'), ! 9 => array('chapter','Chapter questione belongs to'), ! 10 => array('stars','#stars (obsolete)'), ! 11 => array('question','The question itself'), ! 12 => array('answer','Answer to the question'), ! ); //Make sure file is UTF encoded with BOM and has MAC linefeeds $file_path = ! DRUPAL_ROOT.'/'.drupal_get_path('module', 'vtmigrate').'/vragen.csv'; $this->source = new MigrateSourceCSV($file_path, ! $columns, ! array( ! ! 'header_rows' => 1, ! ! 'delimiter' => ';', ! ) );
  • 21. MAPPING - PART 1 ! //map table $this->map=new MigrateSQLMap($this->machineName, ! array( ! ! 'idnr' =>array( ! ! ! 'type' => 'int', ! ! ! //'length' => 11, ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The id of the question', ! ! ),! ! ), ! MigrateDestinationNode::getKeySchema() ); ! //destination $this->destination = new MigrateDestinationNode('question');
  • 22. MAPPING - PART 2 //destination $this->destination = new MigrateDestinationNode('question'); $this->addFieldMapping('title','title'); $this->addFieldMapping('body','question') ! ->arguments( ! ! array( ! ! ! 'format' => 'full_html', ! ! ) ! ); $this->addFieldMapping('field_answer','answer') ! ->arguments( ! ! array( ! ! ! 'format' => 'full_html', ! ! ) ! ); $this->addFieldMapping('created','date'); $this->addFieldMapping('field_course_term','course');
  • 23. MAPPING - PART 2 //skip these source fields $this->addUnmigratedSources( ! array( ! ! 'year', ! ! 'questionnr', ! ! 'stars', ! ! 'time', ! ! 'course', ! ! 'chapter', ! ! //'answer', ! ! 'responder', ! ! 'mnr', ! ) ); //skip these destination fields $this->addUnmigratedDestinations( ! array( ! ! 'revision_uid', ! ! 'log', ! ! 'comment', !! ! //¡­ ! ! 'language', ! ! 'path', ! ) );
  • 26. READY TO IMPORT - OPTIONS
  • 27. DRUPALCAMP 2014 GHENT VARIOUS TOPICS
  • 29. DEPENDENCIES //add dependency $this->dependencies = array('AllChapterInOne');
  • 30. DRUSH Better/faster than UI Start import Reasons why you should drush
  • 31. DRUSH drush mi question --limit=¡°100 items¡± drush mi --rollback drush ms question
  • 32. PREPAREROW public function prepareRow($row){ ! //setting creationtime to correct timestamp ! $row->date = $this->convertDateToTimestamp($row->date,$row->time,$row); ! return TRUE; }
  • 33. PREPARE function prepare($question,stdClass $row){ ! $question->body['und'][0]['value_format'] = 'filtered_html'; !$question->body['und'][0]['format'] = 'filtered_html'; ! !$question->field_answer['und'][0]['value_format'] = 'filtered_html'; !$question->field_answer['und'][0]['format'] = 'filtered_html'; }
  • 34. MULTI-KEY MAP //map table $this->map=new MigrateSQLMap($this->machineName, ! array( ! ! 'chapterid' =>array( ! ! ! 'type' => 'int', ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The chapter ID', ! ! ), ! ! 'course' => array( ! ! ! 'type' => 'varchar', ! ! ! 'length' => 10, ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The course ID', ! ! ), ! ! 'year' => array( ! ! ! 'type' => 'int', ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The year of the course', ! ! ),! ! ), ! MigrateDestinationTerm::getKeySchema() );
  • 35. USER IMPORT class UserMigration extends Migration { ! public function __construct() { // Always call the parent constructor first for basic setup parent::__construct(); $this->description = t('Migrate users from the source database to drupal users'); //map table $this->map=new MigrateSQLMap($this->machineName, ! array( ! ! 'unr' =>array( ! ! ! 'type' => 'varchar', ! ! ! 'length' => 8, ! ! ! 'not null' => TRUE, ! ! ! 'description' => 'The KU Leuven unumber', ! ! ),! ! ), ! MigrateDestinationUser::getKeySchema() ); //¡­
  • 36. USER IMPORT ! //source definition $columns = array( ! 0 => array('name','Name of the user'), ! 1 => array('unr','Unumber of the user'), ! 2 => array('email','Email of the user'), ! 3 => array('course','The course the user is involved'), ! 4 => array('monitor','Is this user a monitor?'), ! 5 => array('id','Database id'), ! 6 => array('roles','The courses involved (will be mapped to roles)'), ); //Make sure file is UTF encoded with BOM and has MAC linefeeds $file_path = ! DRUPAL_ROOT.'/'.drupal_get_path('module', 'vtmigrate').'/personeel.csv'; //drupal_set_message(fopen($file_path,'r')); //drupal_set_message($file_path); $this->source = new MigrateSourceCSV($file_path, ! $columns, ! array( ! ! 'header_rows' => 1, ! ! 'delimiter' => ';', ! ) ); //destination $this->destination = new MigrateDestinationUser(); //¡­
  • 37. USER IMPORT - MAPPING //mapping $this->addFieldMapping('name','unr'); $this->addFieldMapping('mail','email'); //$this->addFieldMapping('role_names','roles') //!! ->separator(','); //skip these source fields $this->addUnmigratedSources( ! array( ! ! 'name', ! ! 'course', ! ! 'monitor', ! ! 'id', ! ! 'roles', ! ) ); //skip these destination fields $this->addUnmigratedDestinations( ! array( ! ! 'is_new', ! ! //¡­ ! ! 'picture', ! ! 'init', ! ! 'path', ! ! 'pathauto', ! ) );
  • 38. USER MAPPING - PREPARE public function prepare($user,stdClass $row){ ! //add all the role ids to the user account ! $role_names = explode(',',$row->roles); ! ! //check if role didactisch team bestaat ! $new_role = array('didactisch team'); ! foreach($new_role as $r){ ! ! if(!user_role_load_by_name($r)){ ! ! ! //create it ! ! ! $role = new stdClass(); ! ! ! $role->name = $r; ! ! ! user_role_save($role); ! ! } ! } ! $role = user_role_load_by_name('didactisch team'); ! //$user role is a mapping from role id to role name ! $user->roles[$role->rid] = $role->name; }
  • 39. DATE //as suggested in https://drupal.org/node/1806296#comment-6589830 public function prepareRow($row) { ! //for some reason, 2 hours have to be added ! $row->startdate = strtotime("+2 hours",$row->startdate); ! $row->enddate = strtotime("+2 hours",$row->enddate); ! $row->datespan = drupal_json_encode(array( ! ! 'from' !=> $row->startdate, ! ! 'to' ! => $row->enddate ! )); ! //change linebreaks back to newlines ! $row->body = str_replace('<br/>',"n",$row->body); }
  • 40. REFERENCE public function prepareRow($entity,stdClass $row) { ! //¡­ ! $loc_query = db_select('node','n') ! ! ->fields('n') ! ! ->condition('n.type','location') //MYSQL's like is case insensitive ! ! ->condition('n.title','%'.$key.'%','LIKE'); ! $locations = $loc_query->execute(); ! ! ! ! ! if(!empty($locations)){ ! ! foreach($locations as $loc){ ! ! //target_id is specific for entity reference ! ! // cf https://drupal.org/node/1845986 ! ! $entity->field_location['und'][0]['target_id'] = $loc->nid; ! } }
  • 41. USER MIGRATION $this->destination = new MigrateDestinationUser( ! array( ! 'md5_passwords' => TRUE, ) );
  • 42. FILES & IMAGES Link to files in answer Be creative Use Regex :)
  • 43. FILES & IMAGES $this->addFieldMapping('field_attachments','files') ! ->separator('#'); $this->addFieldMapping('field_images','images') ! ->separator('#');
  • 44. FILES & IMAGES $files = array(); $nr_matches = preg_match_all('/<a href="(S+)"/i',$row->answer,$matches,PREG_SET_ORDER); //drupal_set_message("#match: ".print_r($nr_matches,true),'warning'); if($nr_matches > 0){ drupal_set_message("#filematch: ".print_r($matches,true),'warning'); ! $base_url = 'https://mirw.kuleuven.be/vragentrommel/trommelroot/'; ! //all matches are stored in an array at $matches[0] foreach($matches as $match){ ! $files[] = $base_url .$match[1]; ! } } //add the URI of the files to $row->files and separate them by hashtag $row->files = implode('#',$files);
  • 45. FILES & IMAGES $images = array(); $nr_matches = preg_match_all('/<img src=/slideshow/add-the-migrate-module-to-your-toolbox/41309300/([^s>]+)>/i& $matches,PREG_SET_ORDER); //drupal_set_message("#match: ".print_r($nr_matches,true),'warning'); if($nr_matches > 0){ drupal_set_message("#image match: ".print_r($matches,true),'warning'); ! $base_url = 'https://mirw.kuleuven.be/vragentrommel/trommelroot/'; ! //all matches are stored in an array at $matches[0] ! foreach($matches as $match){ ! $file_url = $base_url . $match[1]; ! ! $images[] = $file_url; ! } } //add the URI of the images to $row->images, separated with # $row->images = implode('#',$images);
  • 46. DRUPALCAMP 2014 GHENT CONCLUSION
  • 48. RESOURCES Migrate Documentation Migrate Example BTMash
  • 49. Q & A @pjdrouillon