Ticket #122 (closed defect: duplicate)

Opened 11 months ago

Last modified 2 months ago

ffmpeg / vpx-analyse miss

Reported by: MC-arjen Owned by: rklein
Priority: major Milestone: MediaMosa 2.1
Component: Core Version: 2.0. transcoding package
Keywords: Cc:
MoSCoW: Must Have Estimated time after impact analysis:
Related to project: none Tested: yes
Accepted: yes Estimated Hours:

Description

There is an mp4 file which is not correctly analysed by vpx-analyse:

$ /var/opt/www/vpx/htdocs/lua/vpx-analyse /mnt/naspcd1/vpx-acc/data Xu4NkC99gKrbXkH9oN81hiWl
MIME-type: application/octet-stream
ffmpeg-output: FFmpeg version 0.5, Copyright (c) 2000-2009 Fabrice Bellard, et al.}-{ configuration: --enable-libx264 --enable-gpl --enable-pthreads --enable-libmp3lame --enable-libfaac --enable-libfaad --enable-libvorbis --enable-libtheora --enable-libgsm --enable-libxvid}-{ libavutil 49.15. 0 / 49.15. 0}-{ libavcodec 52.20. 0 / 52.20. 0}-{ libavformat 52.31. 0 / 52.31. 0}-{ libavdevice 52. 1. 0 / 52. 1. 0}-{ built on Jun 25 2009 10:43:55, gcc: 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)}-{}-{Seems stream 0 codec frame rate differs from container frame rate: 60000.00 (60000/1) -> 30.00 (30/1)}-{Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/mnt/naspcd1/vpx-acc/data/X/Xu4NkC99gKrbXkH9oN81hiWl':}-{ Duration: 00:01:25.43, start: 0.-00033, bitrate: 10440 kb/s}-{ Stream #0.0(jpn): Video: h264, yuv420p, 1280x720, 30 tbr, 30k tbn, 60k tbc}-{ Stream #0.1(jpn): Audio: aac, 44100 Hz, mono, s16}-{At least one output file must be specified

While ffmpeg -i gives:
$ ffmpeg -i X/Xu4NkC99gKrbXkH9oN81hiWl
FFmpeg version 0.5, Copyright (c) 2000-2009 Fabrice Bellard, et al.

configuration: --enable-libx264 --enable-gpl --enable-pthreads --enable-libmp3lame --enable-libfaac --enable-libfaad --enable-libvorbis --enable-libtheora --enable-libgsm --enable-libxvid
libavutil 49.15. 0 / 49.15. 0
libavcodec 52.20. 0 / 52.20. 0
libavformat 52.31. 0 / 52.31. 0
libavdevice 52. 1. 0 / 52. 1. 0
built on Jun 25 2009 10:43:55, gcc: 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

Seems stream 0 codec frame rate differs from container frame rate: 60000.00 (60000/1) -> 30.00 (30/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'X/Xu4NkC99gKrbXkH9oN81hiWl':

Duration: 00:01:25.43, start: 0.-00033, bitrate: 10440 kb/s

Stream #0.0(jpn): Video: h264, yuv420p, 1280x720, 30 tbr, 30k tbn, 60k tbc
Stream #0.1(jpn): Audio: aac, 44100 Hz, mono, s16

At least one output file must be specified

Could very well be an issue with the start: 0.-00033,

Change History

Changed 11 months ago by forgacs

  • owner set to forgacs
  • status changed from new to assigned

Changed 11 months ago by forgacs

  • owner changed from forgacs to MC-arjen

The problem was the MIME-type: application/octet-stream.
It is solved now.

Changed 11 months ago by Frans

  • related_to set to none
  • milestone changed from MediaMosa X.X to MediaMosa 1.6

Great. Would like to see this backported in the next 1.6.1 release.

Changed 10 months ago by Michiel.Schok

Just uploaded asset XhWImLcQ6tIeLtWt9fcoofrm / mediafile 8u7wg8KbdBr2DEB5qQsE3rv7 to Core 1.6.1
It's detected as h.264, container type is set, but mime-type is still application/octet-stream.

That's not good!

Changed 10 months ago by MC-arjen

This is expected, we didnt fix the 'file' tool, only made sure that the analyse script still does a correct analyse regardless of the detected mimetype. You should check that the file has correct technical metadata and is playable...

Changed 10 months ago by Michiel.Schok 

/asset/XhWImLcQ6tIeLtWt9fcoofrm/play?user_id=foo&mediafile_id=8u7wg8KbdBr2DEB5qQsE3rv7&response=object

works great, and returns a playable object.

BUT... SURFmedia implements a filter on 'mime-type' to determine which mediafiles are shown to the user, and which not. We have implemented an 'allowed list' and add all kind of 'media types' to it. But I hesitate to add 'application/octet-stream' to make it playable...

Changed 10 months ago by Michiel.Schok

Om ook video's van een HD-flip te herkennen moet er een regel toegevoegd
worden aan de magic.mime file en de database opnieuw gegenereerd worden.
Op de meeste systemen moet dit gebeuren met root-rechten.

1. Zoek de volgende regel in magic.mime (staat meestal in /usr/share/file/):

4 string ftyp

Voeg daar_achter_ de volgende regel toe:

8 string MSNV video/mp4

2. Genereer de de database opnieuw. Tik in een shell met rootrechten in:

file -Ci

Changed 10 months ago by Michiel.Schok

Copy/paste haalde een > weg.
De juiste regel die toegevoegd moet worden is

8 string MSNV video/mp4

Changed 10 months ago by Michiel.Schok

Driemaal is scheepsrecht:

> 8      string          MSNV          video/mp4

Changed 10 months ago by Jens

Dependent on the version of 'file' used, it may be necessary to alter /usr/share/file/magic instead of /usr/share/file/magic.mime. To be sure, also add

>8      string          MSNV            \b, MPEG v4 system, PSP
!:mime  video/mp4

after the line

4       string          ftyp            ISO Media

in /usr/share/file/magic. I'll try commit this modification to the maintainers of file to make sure this stays fixed with updated versions of file.

Changed 10 months ago by MC-arjen

ok, will do. The VP-Core servers use a debian with a /etc/magic.mime in which normally these kind of additions are added. i'll have a further look at this.

(Tevens zou ik wil voorstellen om aan ZX te vragen om een latere versie van de magiclib te installeren, bijvoorbeeld van een debian backport, dan zijn we ook gegarandeerd van andere nieuwe bestandsdefinities.)

Changed 10 months ago by Frans

  • version set to Not Accepted

Submitted request for ZX to the Servicedesk@Kennisnet

Changed 9 months ago by MC-arjen

  • status changed from assigned to closed
  • resolution set to fixed

We added the mp4 definition to /etc/magic on the transcoding servers. (The backport of newer file/magic versions will have a followup in vp-core-trac, as this doesnt belong here)

Changed 9 months ago by Michiel.Schok

  • status changed from closed to reopened
  • resolution fixed deleted

Output before re-analysis:

<metadata>
<video_codec>h264</video_codec>
<colorspace>yuv420p</colorspace>
<width>1280</width>
<height>720</height>
<fps>30</fps>
<audio_codec>aac</audio_codec>
<sample_rate>44100</sample_rate>
<channels>mono</channels>
<file_duration/>
<container_type>mov;mp4;m4a;3gp;3g2;mj2</container_type>
<bitrate>0</bitrate>
<bpp/>
<filesize>16231726</filesize>
<mime_type>application/octet-stream</mime_type>
<created>2009-10-06 11:21:17</created>
<changed>2009-10-06 11:21:17</changed>
<is_hinted>TRUE</is_hinted>
<is_inserted_md>FALSE</is_inserted_md>
<still_default>TRUE</still_default>
</metadata>

Output after re-analysis of the file

<metadata>
<video_codec>h264</video_codec>
<colorspace>yuv420p</colorspace>
<width>1280</width>
<height>720</height>
<fps>30</fps>
<audio_codec>aac</audio_codec>
<sample_rate>44100</sample_rate>
<channels>mono</channels>
<file_duration>00:00:11.9</file_duration>
<container_type>mov;mp4;m4a;3gp;3g2;mj2</container_type>
<bitrate>10851</bitrate>
<bpp>0.39</bpp>
<filesize>16231726</filesize>
<mime_type>application/octet-stream</mime_type>
<created>2009-10-20 15:25:17</created>
<changed>2009-10-20 15:25:17</changed>
<is_hinted>TRUE</is_hinted>
<is_inserted_md>FALSE</is_inserted_md>
<still_default>TRUE</still_default>
</metadata>

While additional metadata has been found (length, bitrate), the mime-type is still 'application/octet-stream' instead of 'video/mp4'

Not accepted, reopen

Changed 9 months ago by Frans

Just adding the mp4 definition to /etc/magic on the transcoding servers is not enough as it is only the human readable version. I trust you did generate the real magic file /etc/magic.mgc with command 'file -C; file -Ci'?

Or should this be an action for ZX?

Changed 9 months ago by admin

  • milestone changed from MediaMosa 1.6 to MediaMosa 1.7

Changed 9 months ago by Frans

Related to http://vpcore-trac.madcap.nl/ticket/45
Waiting for feedback from Kennisnet.

Changed 9 months ago by Frans

  • owner changed from MC-arjen to rklein
  • status changed from reopened to assigned

Changed 8 months ago by MC-arjen

  • status changed from assigned to closed
  • resolution set to fixed

Op acceptatie ziet het er nu ook goed uit:

@app1:/mnt/naspcd1/vpx-acc/data$ file 8/8u7wg8KbdBr2DEB5qQsE3rv7
8/8u7wg8KbdBr2DEB5qQsE3rv7: video/mp4

@transcoding2-linux:/mnt/naspcd1/vpx-acc/data$ file 8/8u7wg8KbdBr2DEB5qQsE3rv7
8/8u7wg8KbdBr2DEB5qQsE3rv7: video/mp4

@transcoding1-linux:/mnt/naspcd1/vpx-acc/data$ file 8/8u7wg8KbdBr2DEB5qQsE3rv7
8/8u7wg8KbdBr2DEB5qQsE3rv7: video/mp4

Er zit echter wel verschil tussen prod en acc:

acceptatie:
@transcoding1-linux:/mnt/naspcd1/vpx-acc/data$ dpkg -l | grep magic
ii file 4.26-1 Determines file type using "magic" numbers
ii libmagic1 4.26-1 File type determination library using "magic

prod:
$ dpkg -l | grep magic
ii file 4.17-5etch3 Determines file type using "magic" numbers
ii libmagic1 4.17-5etch3 File type determination library using "magic

Hopelijk wordt dit snel opgelost...

Changed 8 months ago by Frans

  • version changed from Not Accepted to 1.7.1

Changed 8 months ago by Michiel.Schok

  • status changed from closed to reopened
  • resolution fixed deleted

Jens heeft aan asset JY7yGqXn7YISqOCvXU3Tasf1 mediafile W2Tr0n1An3JE8zAcNXBYp4WZ geupload.

De metadata van het origineel laat nog steeds application/octet-stream zien bij de mediafile.

[GET] mediafile/W2Tr0n1An3JE8zAcNXBYp4WZ

...
<metadata>
        <video_codec>h264</video_codec>
        <colorspace>yuv420p</colorspace>
        <width>1280</width>
        <height>720</height>
        <fps>30</fps>
        <audio_codec>aac</audio_codec>
        <sample_rate>44100</sample_rate>
        <channels>mono</channels>
        <file_duration>00:00:50.0</file_duration>
        <container_type>mov;mp4;m4a;3gp;3g2;mj2</container_type>
        <bitrate>0</bitrate>
        <bpp></bpp>
        <filesize>4457949</filesize>
        <mime_type>application/octet-stream</mime_type>
        <created>2009-11-20 11:26:06</created>
        <changed>2009-11-20 13:29:21</changed>
        <is_hinted>FALSE</is_hinted>
        <is_inserted_md>FALSE</is_inserted_md>
      </metadata>

Changed 7 months ago by Frans

  • milestone changed from MediaMosa 1.7 to MediaMosa 2.1

Moved to MediaMosa 2.1

Changed 7 months ago by Frans

  • moscow set to Must Have

Changed 7 months ago by Frans

debian package incl. new mimetype libs.

Changed 7 months ago by MC-arjen

  • version changed from 1.7.1 to 2.0. transcoding package

Changed 2 months ago by Frans

  • status changed from reopened to closed
  • resolution set to duplicate

Duplicate of Mime package (lib magic).
Can be closed.

Changed 2 months ago by Michiel.Schok

  • tested set to yes
  • accepted changed from no to yes

new analyse of the video shows a mime-type of 'video/mp4'.
Thats great!

Note: See TracTickets for help on using tickets.