Unable to open from external storage (as file descriptor)?

[earwole1]

On Pixel2, I have taken a few short videos. 5-10 seconds each.

Within PathUtil.getPath(), the URI is:
content://com.android.externalstorage.documents/document/primary%3ADCIM%2FCamera%2FVID_20200124_154910.mp4

the check for isExternalStorageDocument(uri) is True and the variable "String[] split" (line ~35) contains:

• split[0]: primary
• split[1]: DCIM/Camera/VID_20200124_154910.mp4

getPath() then returns: /storage/emulated/0/DCIM/Camera/VID_20200124_154910.mp4

Finally, the C++ code fails within media_file_build():
media_file_builder.cpp:149 invokes onError(jMediaFileBuilder).

So, effectively, every video file I attempt to open uses the pipe interface. I am missing something or is this an issue with the newer (well, new-ish) phones running Android 10?

Thank you for the otherwise awesome repo and the ffmpeg builder.

[Javernaut]

Hello @earwole1 ,

With Android 7 it is forbidden to expose URIs to files in your application with file:// schema.

So the PathUtil.getPath() is just a guessing for the actual path of a local file. Sometimes it is successful, sometimes not. However, the URI should be accessible and we can open a file descriptor for it.

In Android 10 the Scoped Storage was introduced and it greatly restricted the access to files outside of your app. Even a native library can't work with FILE* type in such a case.

So for now I think that pipe protocol is the only way for FFmpeg to access local files on Android 10.

[earwole1]

@Javernaut Thanks for clarifying.

This brings up another question then: why does ffmpeg not always produce all frames request? With the pipe protocol, and the default request of 4 frames,
sample_encoding.txt
the application provides between 1 and 4 frames[1], depending on the video provided. Ultimately the call to av_read_frame() in frame_extractor.cpp will fail.

The videos I'm using were acquired by the device. Sample encoding is attached. I'd be happy to close this issue and open another one with additional logging. Ultimately, all I'm interested in is dumping every video frame to the filesystem.

Thanks!
[1] I removed the seek to the "middle" of the video. If I keep that code in place then I receive either 0 or 1 decoded frame.

[Javernaut]

@earwole1 , I have only a part of an answer now.

Currently I don't have enough time to dive into this issue as I want to.

What I know now:

• The problem can be because of improper seeking. Sometimes the av_seek_frame method fails and thus the reading itself fails too. This happens because of the av_seek_frame looks for key frames by default and sometimes there is no such a frame in the place where the app tries to seek. Moreover, in a very short video there may be only one single key frame. I personally faced this situation.
• According to FFmpeg, the pipe protocol may not work properly with mov (mp4) file format, which is quite sad, as it is perhaps the most popular format in Android. So sometimes the whole decoding of the same file works if it is accessed by file:// and doesn't work in case of pipe: or works partially.
• Recently I got another idea - to use Android's standard implementation of h264 decoder. Perhaps it will do better. It is just a guess, as it is a decoder and not the demuxer.

This is all I have in mind for now. Once I have time, I'll return to this whole thing as it bothers me a lot.

[earwole1]

@Javernaut Again, thanks for the quick reply. I'll keep looking at the issue and if I find a solution I'll submit a PR.

[YiChaoLove]

• About av_seek_frame fail, @Javernaut answer is correct, but we can continue to decode the following frames through the I frame until we find the pts we want, there are related questions on stackoverflow, ffmpeg av_seek_frame with AVSEEK_FLAG_ANY causes grey screen, and this is my demo Forked from What the Codec.

• About pipe protocol, we need to pay attention to the size of the pipe buffer and the position of the moov in the mp4 (mov) file. In the demo I have explained how to use pipe protocol.

• About Android MediaCodec, although MediaCodec has high performance, it has poor compatibility.

Thanks @Javernaut code and article(A Story about FFmpeg on Android).