Large Scale Recurring Imports in Drupal 8: Architecture and Implementation Tips

Nowadays Drupal is not simply a CMS, it's a big and complicated framework that stands behind large enterprise level websites.

Drupal is used successfully for government, high education and healthcare websites that store large volumes of data. Often enough these organizations need their data to be updated over night, monthly or annually. It can be as simple as stock information updates performed once an hour and as big as update of all programs and courses offered by the college and university including all of the program details and costs.

Large scale complicated imports bring their challenges: time and hosting resources, parsing algorithms, different sources: XML, JSON, CSV. In my talk I'll cover some problems that a developer may face while building imports of data from various external sources into Drupal. I'll cover different formats such as XML, CSV and JSON, as well as approaches that can be used to make the task easier.

The audience will learn how to build imports with solid architecture so they would have less problems with performance as well as won't be limited by cron job time. I will also cover Batch and Queue APIs in Drupal 8 as well as touch base on continuous integration tool such as Jenkins.

The talk is for intermediate to advanced level back-end developers.

Experience level: 
Intermediate
Speaker(s): 
amykhailova

Date and time

Friday, September 22, 2017 - 13:25 to 13:55
Track: 

Session Length

30:30 minutes

Room

5th Floor Room 501 - Beyond Spots and Dots

Role

Back end developer / Systems admin

Drupal version

Drupal 8

Bridges

Education
Business
Non-profit