Ticket #147 (closed enhancement: fixed)

Opened 4 years ago

Last modified 3 years ago

Support Callback URL after Transcoding job

Reported by: thijs@… Owned by: MC-Arjen
Priority: major Milestone: MediaMosa 1.7
Component: Core Version: 1.7.0
Keywords: callback, transcoding, job Cc:
MoSCoW: Must Have Estimated time after impact analysis:
Related to project: Rich Media Tested: yes
Accepted: yes Estimated Hours:

Description

With adding a transcoding job to MediaMosa it should be possible to provide a callback URL which will be called after a status change in the transcoding job.

The callback URL should be called with some POST parameters like job_id/mediafile_id and transcoding status.

After successful transcoding and unsuccesful transcoding this URL should be called allowing the application to directly respond to the finished job instead of the application having to probe MediaMosa every x minutes for a job status check.

Change History

  Changed 4 years ago by MC-arjen

  • owner set to forgacs
  • status changed from new to assigned

/mediafile/upload [post] and [put]

extra param: completed_transcoding_url, value type: url, method: POST, optional.

This url will be called after completion (including analyse) of all transcodings in this request, with the status of the transcoding(s).

same for: /mediafile/{mediafile_id}/transcode [POST]

  Changed 4 years ago by MC-arjen

  • milestone changed from MediaMosa X.X to MediaMosa 1.7

  Changed 4 years ago by thijs@…

Just checking, but wasn't the analysis procedure going to be removed as a separate 'job'?

  Changed 4 years ago by MC-arjen

Correct, we are testing/reviewing this ticket at the moment (#160)

  Changed 4 years ago by forgacs

  • owner changed from forgacs to robert

The development is ready. Robert has to review the code.

  Changed 4 years ago by forgacs

Robert: wget generates tmp files to my Mediamosa root. As I see, it shouldn't be. I think it is just the settings of my computer. Please, check it on your environment.
Source: sites/all/modules/vpx_jobs/vpx_jobs.module vpx_jobs_notify_transcoding() function, line: 1063.
Code: exec('wget --quiet ' . escapeshellcmd($completed_transcoding_url . $new_status .' >/dev/null 2>/dev/null &'));
Thanks in advance.

  Changed 4 years ago by forgacs

@Robert: Tmp bug is fixed.

  Changed 4 years ago by robert

  • status changed from assigned to closed
  • resolution set to ready for review

  Changed 4 years ago by robert

  • status changed from closed to reopened
  • resolution ready_for_review deleted

  Changed 4 years ago by robert

  • status changed from reopened to closed
  • resolution set to fixed

  Changed 4 years ago by Frans

  • version set to 1.7.0

  Changed 4 years ago by Frans

(MC-Arjen) When a transcoding job finishes, a ega defined url is called. No more
polling for jobstatusses is needed anymore in the ega.

  Changed 4 years ago by Michiel.Schok

Tried a transcode callback.

[post] /mediafile/OOkU1W0hs7h0GWlGRKSOuH5i/transcode
[params] completed_transcoding_url=http://richmedia.showcase.surfnet.nl/core17/transcodefinished.htm&user_id=SURFgroepen.michiel71&profile_id=13

    <item id="1">
      <job_id>43379</job_id>
    </item>

Transcode finishes, but no callback is seen.
Documentations is not up-to-date at mediamosa.org

follow-up: ↓ 16   Changed 4 years ago by forgacs

Documentation is updated.
I have tested the completed_transcode_url and seems fine. Would you paste here the content of transcodefinished.htm?

  Changed 4 years ago by Michiel.Schok

Also tried a callback after trancoding using a small form.
HTML is as follows:

<html>
<body>
<form method="post" enctype="multipart/form-data" action="http://upload.acceptatie.vpx.kennisnet.nl/mediafile/upload?upload_ticket=P00rgVzMw8sKKQw8kqtTsfCE" />
<input type="file" name="file" />
<input type="hidden" name="redirect_uri" value="http://richmedia.showcase.surfnet.nl/core17/redirect_after_upload.htm" />
<input type="hidden" name="completed_transcoding_url" value="http://richmedia.showcase.surfnet.nl/core17/completed_transcode.htm" />
<input type="hidden" name="completed_url" value="http://richmedia.showcase.surfnet.nl/core17/completed_upload.htm" />
<input name="transcode[]" value="13" type="hidden"/>
<input type="submit" value="Upload" />
</form>
</body>
</html>

In de logfiles of the webserver, I notice

2009-11-09 15:27:02 W3SVC1 192.87.116.69 GET /core17/upload.htm - 80 - 192.87.109.229 Mozilla/5.0+(Windows;+U;+Windows+NT+6.1;+en-US;+rv:1.9.1.3)+Gecko/20090824+Firefox/3.5.3 200 0 0
2009-11-09 15:27:16 W3SVC1 192.87.116.69 GET /core17/completed_upload.htm upload_ticket=P00rgVzMw8sKKQw8kqtTsfCE&status_code=601 80 - 145.97.33.213 Wget/1.11.4 404 0 2
2009-11-09 15:27:16 W3SVC1 192.87.116.69 GET /core17/redirect_after_upload.htm - 80 - 192.87.109.229 Mozilla/5.0+(Windows;+U;+Windows+NT+6.1;+en-US;+rv:1.9.1.3)+Gecko/20090824+Firefox/3.5.3 404 0 2

So VP-core correctly notifies the finish of an upload, the client gets a successfull redirect to the new page. But there is no callback on finish of the transcode.
In the management portal, I can see the start and successful finish of the transcode job. It's ID 43401 

server	job 43401 klaar, nieuwe mediafile opslaan als: /mnt/naspcd1/vpx-acc/data/V/V4NxGkF9ldedJsDPHTGtCpnz	notice	11/09/2009 - 16:28
server	Einde TRANSCODE job: 43401 met status: FINISHED	notice	11/09/2009 - 16:28
server	About to start TRANSCODE job: 43401 calling exec: /opt/www/transcoding/htdocs/lua/vpx-transcode /mnt/naspcd1/vpx-acc/data Ggj4kH8pTBjtMmncsiOeMsRb flv '-ab 40000 -acodec libmp3lame -ar 22050 -r 15 -b 200000 -vcodec flv -s 352x200 -padtop 44 -padbottom 44 ' > /dev/null &	notice	11/09/2009 - 16:27
server	Started TRANSCODE job	notice	11/09/2009 - 16:27
jobhandler	Start job: jobid=43401, job_type=TRANSCODE, serverip=10.252.0.177, mediafile_id=Ggj4kH8pTBjtMmncsiOeMsRb	notice	11/09/2009 - 16:27
jobhandler	Start job:http://10.252.0.177/internal/server/jobstart?job_id=43401&job_type=TRANSCODE&mediafile_src=Ggj4kH8pTBjtMmncsiOeMsRb&tool=ffmpeg&file_extension=flv&command=-ab:40000;;-acodec:libmp3lame;-ar:22050;;-r:15;;;;;;-b:200000;-vcodec:flv;-s:352x200;-padtop:44;-padbottom:44&testtag=FALSE	notice	11/09/2009 - 16:27
jobhandler	Stuur TRANSCODE job: 43401 naar server op: 10.252.0.177	notice	11/09/2009 - 16:27

in reply to: ↑ 14   Changed 4 years ago by Michiel.Schok

Replying to forgacs:

Documentation is updated.
I have tested the completed_transcode_url and seems fine. Would you paste here the content of transcodefinished.htm?

sure, it's online and says

<html>
it's ready
</html>

 http://richmedia.showcase.surfnet.nl/core17/transcodefinished.htm

  Changed 4 years ago by Michiel.Schok

  • status changed from closed to reopened
  • resolution fixed deleted

  Changed 4 years ago by MC-arjen

  • owner changed from robert to marc
  • status changed from reopened to assigned

The transcoding server has no access to the outside world, only the app servers have.

Can we request a firewall change, or do we have to rewrite the solution (something with a hub to the app server...) Request answer of KN

  Changed 4 years ago by MC-arjen

  • owner changed from marc to frans

no answer, but in general i think we should leave the transcoding servers behind the firewall, and implement a /internal rest call on app1/app2 to forward a completed_transcode_url to the outside world. 1.7.2 would be nice. reassign to Frans for feedback.

  Changed 4 years ago by Frans

  • owner changed from frans to MC-Arjen

1.7.2. would be nice, but I think there are more important issues.
(nice to have).

The proposed internal restcall would be the way to go.

  Changed 4 years ago by Frans

Agreed to open FW for port 80- outgoing on the transcoding servers.
Submitted to VP-Core trac #...

  Changed 4 years ago by Frans

Action ZX:  http://vpcore.trac.madcap.nl/ticket/85

  Changed 3 years ago by Frans

  • status changed from assigned to closed
  • tested set to yes
  • moscow set to Must Have
  • accepted changed from no to yes
  • resolution set to fixed

Is realized...

Note: See TracTickets for help on using tickets.