Sync Media with MediaMosa

[robert]

Metadata input is stored on the drupal side, using the Media module. This data however is also stored on MediaMosa.

Only one should be considered leading. That means either the data in MediaMosa is leading or the data in front end Drupal Media object is leading.

In my opinion, MediaMosa metadata is leading. Any changes made to the Media object must be synced to the MediaMosa storage. Any changes made in MediaMosa must be synced back to the Media object.

That means that we have 2-way sync; Drupal Media updates->MediaMosa and MediaMosa->Drupal Media.

This gives one problem; when do we sync Metadata from MediaMosa to Drupal Media? Also, the user will need to re-connect or re-sync with MediaMosa. This is because the Drupal side will go out of sync with MediaMosa (MediaMosa database can be altered).

The scope of the ticket;

• Sync metadata to MediaMosa when a metadata value is added.
• Sync metadata to MediaMosa when a metadata value is changed.
• Sync metadata from MediaMosa at certain intervals or by user action.

Syncing is done using the stored URI in the Media object. This will tell you if the object is MediaMosa and figure out the asset_id. At time of writing, the url is build as  mediamosa://mediafile/[mediafile_id]/[filename]. This might change to include the asset_id

Syncing should be something like

• getting all media objects
• Is it mediamosa object?
  • yes;
    • Is data drupal media out of sync with MediaMosa (compare with videotimestampmodified)
      • get metadata from mediamosa->write to drupal.
      • store the next videotimestampmodified in drupal so we know its (un)changed next sync run.
  • no; next object
  • object deleted on side of MediaMosa; delete on drupal side.
• getting all mediamosa assets
  • Any new assets?
    • yes; add to drupal with metadata.
    • no; next asset.

When getting all assets from MediaMosa; use created date;

• Get all assets created >= 'my stored oldest created date' (first time will be all assets).
• Loop through the new assets and add the missing assets to Drupal.
• At the end you have to store the oldest created date you found and add 1 second. Next time you'll use that date to get the next new assets from MediaMosa.
• The first time you'll won't have a created date, so get all the assets->store the oldest created date+1 second.
• The oldest created date must be stored somewhere, thinking about a drupal variable. Watch it; the name of the variable should be linked to the MediaMosa connector information. Why? Because the user might change from client application, or even MediaMosa server and then this created value will not belong to the same MediaMosa database. I'm thinking about using the URL of the MediaMosa connector (mediamosa_connector_url) and the mediamosa_connector_username to create a unique value (user_name@url). Then use sha1(user_name@url) to get a unique value from the 2 strings combined and store the variable as 'mediamosa_sync_[sha1 number]'. So each time you get the correct created date for the correct mediamosa connector, not matter if the user decided to change connection. This means that the user will need to re-sync, but will fix that later. For now the user will need to do this himself by pressing the re-sync button.
• Any information stored that is linked to the mediamosa connection will need to be stored as an similar way to be unique per connection.

I think this should cover 2-way sync. Its important to remember that data in MediaMosa is leading and Drupal Media metadata is secondary. Sync to MediaMosa when Drupal Media is edited. There is no auto-sync from MediaMosa to Drupal Media. This might be something MediaMosa can do in the future, but not now.

The user must be able to start the sync with a button somewhere, i'm thinking on new page, next to configuration (tab config/tab sync). On the sync page it should show when the last sync took place. Store the last sync each time the cron has run the sync or the user.

Pressing the button will make the next cron run do the sync. So don't do sync immediately after pressing the button. You never know how much time the sync might take.

Also the sync should be in steps. Syncing 10.000 assets should be done per 100 assets (setting in configuration). So 10.000 / 100 = 1000 cron runs before all are synced.

If the sync is in progress, then this should also be show on the sync page.

Things to remember;

• Syncing is in steps.
• If the syncing is in progress, the user must be able to restart the sync.