Ticket #122 (closed defect: duplicate)

Opened 4 years ago

Last modified 3 years ago

ffmpeg / vpx-analyse miss

Reported by: MC-arjen Owned by: rklein
Priority: major Milestone: MediaMosa 2.1
Component: Core Version: 2.0. transcoding package
Keywords: Cc:
MoSCoW: Must Have Estimated time after impact analysis:
Related to project: none Tested: yes
Accepted: yes Estimated Hours:

Description

There is an mp4 file which is not correctly analysed by vpx-analyse:

$ /var/opt/www/vpx/htdocs/lua/vpx-analyse /mnt/naspcd1/vpx-acc/data Xu4NkC99gKrbXkH9oN81hiWl
MIME-type: application/octet-stream
ffmpeg-output: FFmpeg version 0.5, Copyright (c) 2000-2009 Fabrice Bellard, et al.}-{ configuration: --enable-libx264 --enable-gpl --enable-pthreads --enable-libmp3lame --enable-libfaac --enable-libfaad --enable-libvorbis --enable-libtheora --enable-libgsm --enable-libxvid}-{ libavutil 49.15. 0 / 49.15. 0}-{ libavcodec 52.20. 0 / 52.20. 0}-{ libavformat 52.31. 0 / 52.31. 0}-{ libavdevice 52. 1. 0 / 52. 1. 0}-{ built on Jun 25 2009 10:43:55, gcc: 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)}-{}-{Seems stream 0 codec frame rate differs from container frame rate: 60000.00 (60000/1) -> 30.00 (30/1)}-{Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/mnt/naspcd1/vpx-acc/data/X/Xu4NkC99gKrbXkH9oN81hiWl':}-{ Duration: 00:01:25.43, start: 0.-00033, bitrate: 10440 kb/s}-{ Stream #0.0(jpn): Video: h264, yuv420p, 1280x720, 30 tbr, 30k tbn, 60k tbc}-{ Stream #0.1(jpn): Audio: aac, 44100 Hz, mono, s16}-{At least one output file must be specified

While ffmpeg -i gives:
$ ffmpeg -i X/Xu4NkC99gKrbXkH9oN81hiWl
FFmpeg version 0.5, Copyright (c) 2000-2009 Fabrice Bellard, et al.

configuration: --enable-libx264 --enable-gpl --enable-pthreads --enable-libmp3lame --enable-libfaac --enable-libfaad --enable-libvorbis --enable-libtheora --enable-libgsm --enable-libxvid
libavutil 49.15. 0 / 49.15. 0
libavcodec 52.20. 0 / 52.20. 0
libavformat 52.31. 0 / 52.31. 0
libavdevice 52. 1. 0 / 52. 1. 0
built on Jun 25 2009 10:43:55, gcc: 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

Seems stream 0 codec frame rate differs from container frame rate: 60000.00 (60000/1) -> 30.00 (30/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'X/Xu4NkC99gKrbXkH9oN81hiWl':

Duration: 00:01:25.43, start: 0.-00033, bitrate: 10440 kb/s

Stream #0.0(jpn): Video: h264, yuv420p, 1280x720, 30 tbr, 30k tbn, 60k tbc
Stream #0.1(jpn): Audio: aac, 44100 Hz, mono, s16

At least one output file must be specified

Could very well be an issue with the start: 0.-00033,

Change History

Changed 4 years ago by forgacs

  • owner set to forgacs
  • status changed from new to assigned

Changed 4 years ago by forgacs

  • owner changed from forgacs to MC-arjen

The problem was the MIME-type: application/octet-stream.
It is solved now.

Changed 4 years ago by Frans

  • related_to set to none
  • milestone changed from MediaMosa X.X to MediaMosa 1.6

Great. Would like to see this backported in the next 1.6.1 release.

Changed 4 years ago by Michiel.Schok

Just uploaded asset XhWImLcQ6tIeLtWt9fcoofrm / mediafile 8u7wg8KbdBr2DEB5qQsE3rv7 to Core 1.6.1
It's detected as h.264, container type is set, but mime-type is still application/octet-stream.

That's not good!

Changed 4 years ago by MC-arjen

This is expected, we didnt fix the 'file' tool, only made sure that the analyse script still does a correct analyse regardless of the detected mimetype. You should check that the file has correct technical metadata and is playable...

Changed 4 years ago by Michiel.Schok 

/asset/XhWImLcQ6tIeLtWt9fcoofrm/play?user_id=foo&mediafile_id=8u7wg8KbdBr2DEB5qQsE3rv7&response=object

works great, and returns a playable object.

BUT... SURFmedia implements a filter on 'mime-type' to determine which mediafiles are shown to the user, and which not. We have implemented an 'allowed list' and add all kind of 'media types' to it. But I hesitate to add 'application/octet-stream' to make it playable...

Changed 4 years ago by Michiel.Schok

Om ook video's van een HD-flip te herkennen moet er een regel toegevoegd
worden aan de magic.mime file en de database opnieuw gegenereerd worden.
Op de meeste systemen moet dit gebeuren met root-rechten.

1. Zoek de volgende regel in magic.mime (staat meestal in /usr/share/file/):

4 string ftyp

Voeg daar_achter_ de volgende regel toe:

8 string MSNV video/mp4

2. Genereer de de database opnieuw. Tik in een shell met rootrechten in:

file -Ci

Changed 4 years ago by Michiel.Schok

Copy/paste haalde een > weg.
De juiste regel die toegevoegd moet worden is

8 string MSNV video/mp4

Changed 4 years ago by Michiel.Schok

Driemaal is scheepsrecht:

> 8      string          MSNV          video/mp4

Changed 4 years ago by Jens

Dependent on the version of 'file' used, it may be necessary to alter /usr/share/file/magic instead of /usr/share/file/magic.mime. To be sure, also add

>8      string          MSNV            \b, MPEG v4 system, PSP
!:mime  video/mp4

after the line

4       string          ftyp            ISO Media

in /usr/share/file/magic. I'll try commit this modification to the maintainers of file to make sure this stays fixed with updated versions of file.

Changed 4 years ago by MC-arjen

ok, will do. The VP-Core servers use a debian with a /etc/magic.mime in which normally these kind of additions are added. i'll have a further look at this.

(Tevens zou ik wil voorstellen om aan ZX te vragen om een latere versie van de magiclib te installeren, bijvoorbeeld van een debian backport, dan zijn we ook gegarandeerd van andere nieuwe bestandsdefinities.)

Changed 4 years ago by Frans

  • version set to Not Accepted

Submitted request for ZX to the Servicedesk@Kennisnet

Changed 4 years ago by MC-arjen

  • status changed from assigned to closed
  • resolution set to fixed

We added the mp4 definition to /etc/magic on the transcoding servers. (The backport of newer file/magic versions will have a followup in vp-core-trac, as this doesnt belong here)

Changed 4 years ago by Michiel.Schok

  • status changed from closed to reopened
  • resolution fixed deleted

Output before re-analysis:

<metadata>
<video_codec>h264</video_codec>
<colorspace>yuv420p</colorspace>
<width>1280</width>
<height>720</height>
<fps>30</fps>
<audio_codec>aac</audio_codec>
<sample_rate>44100</sample_rate>
<channels>mono</channels>
<file_duration/>
<container_type>mov;mp4;m4a;3gp;3g2;mj2</container_type>
<bitrate>0</bitrate>
<bpp/>
<filesize>16231726</filesize>
<mime_type>application/octet-stream</mime_type>
<created>2009-10-06 11:21:17</created>
<changed>2009-10-06 11:21:17</changed>
<is_hinted>TRUE</is_hinted>
<is_inserted_md>FALSE</is_inserted_md>
<still_default>TRUE</still_default>
</metadata>

Output after re-analysis of the file

<metadata>
<video_codec>h264</video_codec>
<colorspace>yuv420p</colorspace>
<width>1280</width>
<height>720</height>
<fps>30</fps>
<audio_codec>aac</audio_codec>
<sample_rate>44100</sample_rate>
<channels>mono</channels>
<file_duration>00:00:11.9</file_duration>
<container_type>mov;mp4;m4a;3gp;3g2;mj2</container_type>
<bitrate>10851</bitrate>
<bpp>0.39</bpp>
<filesize>16231726</filesize>
<mime_type>application/octet-stream</mime_type>
<created>2009-10-20 15:25:17</created>
<changed>2009-10-20 15:25:17</changed>
<is_hinted>TRUE</is_hinted>
<is_inserted_md>FALSE</is_inserted_md>
<still_default>TRUE</still_default>
</metadata>

While additional metadata has been found (length, bitrate), the mime-type is still 'application/octet-stream' instead of 'video/mp4'

Not accepted, reopen

Changed 4 years ago by Frans

Just adding the mp4 definition to /etc/magic on the transcoding servers is not enough as it is only the human readable version. I trust you did generate the real magic file /etc/magic.mgc with command 'file -C; file -Ci'?

Or should this be an action for ZX?

Changed 4 years ago by admin

  • milestone changed from MediaMosa 1.6 to MediaMosa 1.7

Changed 4 years ago by Frans

Related to  http://vpcore-trac.madcap.nl/ticket/45
Waiting for feedback from Kennisnet.

Changed 4 years ago by Frans

  • owner changed from MC-arjen to rklein
  • status changed from reopened to assigned

Changed 4 years ago by MC-arjen

  • status changed from assigned to closed
  • resolution set to fixed

Op acceptatie ziet het er nu ook goed uit:

@app1:/mnt/naspcd1/vpx-acc/data$ file 8/8u7wg8KbdBr2DEB5qQsE3rv7
8/8u7wg8KbdBr2DEB5qQsE3rv7: video/mp4

@transcoding2-linux:/mnt/naspcd1/vpx-acc/data$ file 8/8u7wg8KbdBr2DEB5qQsE3rv7
8/8u7wg8KbdBr2DEB5qQsE3rv7: video/mp4

@transcoding1-linux:/mnt/naspcd1/vpx-acc/data$ file 8/8u7wg8KbdBr2DEB5qQsE3rv7
8/8u7wg8KbdBr2DEB5qQsE3rv7: video/mp4

Er zit echter wel verschil tussen prod en acc:

acceptatie:
@transcoding1-linux:/mnt/naspcd1/vpx-acc/data$ dpkg -l | grep magic
ii file 4.26-1 Determines file type using "magic" numbers
ii libmagic1 4.26-1 File type determination library using "magic

prod:
$ dpkg -l | grep magic
ii file 4.17-5etch3 Determines file type using "magic" numbers
ii libmagic1 4.17-5etch3 File type determination library using "magic

Hopelijk wordt dit snel opgelost...

Changed 4 years ago by Frans

  • version changed from Not Accepted to 1.7.1

Changed 3 years ago by Michiel.Schok

  • status changed from closed to reopened
  • resolution fixed deleted

Jens heeft aan asset JY7yGqXn7YISqOCvXU3Tasf1 mediafile W2Tr0n1An3JE8zAcNXBYp4WZ geupload.

De metadata van het origineel laat nog steeds application/octet-stream zien bij de mediafile.

[GET] mediafile/W2Tr0n1An3JE8zAcNXBYp4WZ

...
<metadata>
        <video_codec>h264</video_codec>
        <colorspace>yuv420p</colorspace>
        <width>1280</width>
        <height>720</height>
        <fps>30</fps>
        <audio_codec>aac</audio_codec>
        <sample_rate>44100</sample_rate>
        <channels>mono</channels>
        <file_duration>00:00:50.0</file_duration>
        <container_type>mov;mp4;m4a;3gp;3g2;mj2</container_type>
        <bitrate>0</bitrate>
        <bpp></bpp>
        <filesize>4457949</filesize>
        <mime_type>application/octet-stream</mime_type>
        <created>2009-11-20 11:26:06</created>
        <changed>2009-11-20 13:29:21</changed>
        <is_hinted>FALSE</is_hinted>
        <is_inserted_md>FALSE</is_inserted_md>
      </metadata>

Changed 3 years ago by Frans

  • milestone changed from MediaMosa 1.7 to MediaMosa 2.1

Moved to MediaMosa 2.1

Changed 3 years ago by Frans

  • moscow set to Must Have

Changed 3 years ago by Frans

debian package incl. new mimetype libs.

Changed 3 years ago by MC-arjen

  • version changed from 1.7.1 to 2.0. transcoding package

Changed 3 years ago by Frans

  • status changed from reopened to closed
  • resolution set to duplicate

Duplicate of Mime package (lib magic).
Can be closed.

Changed 3 years ago by Michiel.Schok

  • tested set to yes
  • accepted changed from no to yes

new analyse of the video shows a mime-type of 'video/mp4'.
Thats great!

Note: See TracTickets for help on using tickets.