際際滷

際際滷Share a Scribd company logo

Android media codec ????

?Download as PPTX, PDF?
?
Taehwan kwon
Taehwan kwon
1 of 45
Downloaded 137 times
Android MediaCodec! ???/??? ???! Taehwan (taehwan@thdev.net)
¢ ????? 1? 5?? ¢ ? ?? ???? ??! ?? ??? ??. o http://thdev.net ¢ ??? ?? ???! ¢ NDK ????? ?????. ???
? ? 1. ???/?????? 2. Codec??? a. H.264/AVC b. AAC 3. MediaCodec 4. MediaMuxer 5. ????? 6. Q & A
MediaCodec? ???????? ??, ??? ????? API(MediaRecord)? ??? ...
???? ???? ???? ??!
???? ????
¢ ??? o ????? ??, ??? ????? ?? ???? ?. ¢ ??? o ????? ??, ????? ?? ?????. 1. ???/??? ???
1. H.264 ???/??? ??? ???
Codec? Codec? ??? ??? ????? ??? ??, ????? ???, ?? ? ?? ? ? ?? ????? ????? ????? ?? ??? ?? ??, ??? ??? ??? ???? ?? - ???? : http://ko.wikipedia.org/wiki/%EC%BD%94%EB%8D%B1
H.264/AVC
¢ ITU-T and MPEG? JVT ?? ??? ???? ¢ ITU-T : H.264 ¢ MPEG : MPEG-4 part 10/AVC(Advanced Video Coding) 2. H.264/AVC
¢ ???? Bit-Rate ?? 50% ??? ??? ??? ?? ¢ ?? ?? ?? ¢ ?? ???? ???? ?? 2. H.264/AVC - ??
?Profile : ??? ??? ??? ? ? ¢ Baseline : ??? ???? ?? ¢ Main : ?? ? ??? ¢ Extended : ??? ????? ?? 2. H.264/AVC - Profile
¢ I-Frame(Intra-frame) ¢ P-Frame(Predictive-frame) ¢ B-Frame(Bi-directionally-frame) 2. H.264/AVC - Frame
¢ I-Frame(Intra-frame) o ?? ??? ?? ????? Main? ?? 2. H.264/AVC - Frame
¢ P-Frame(Predictive-frame) o ?? ???? ?? ????? ?? ? ?? ??? ???. ¢ B-Frame(Bi-directionally-frame) o ???/??? ??? ??? ??? ?? ??. o ???? ???? ?? ??. 2. H.264/AVC - Frame
¢ H.264? YUV420 ???? ??. o YUV? ?? ??, ????? ???? ??. o YUV? ???? ??? ?? TV?? ??? ??. 2. H.264/AVC - ???
¢ Android? 2?? YUV ?? ? ??. o I420, NV12 o ??? : width * height * 3 / 2 o ?? : YUV ??? ?? 2. H.264/AVC - YUV ???
¢ ?? ?? ???? ?? bit ? (bit/sec) ¢ ? bitrate? ????? ???. 2. H.264/AVC - Bitrate
¢ FPS(Frame per second) o 1?? ????? ? ??? ?? ???. ¢ MediaCodec - Frame rate 2. H.264/AVC - Frame rate(FPS)
I-Frame ?? ??(GOP : Group Of Picture) ¢ I-Frame? I-Frame ??? ?? o I P P P P I : 5 ¢ MediaCodec - I Frame Interval o ? ??? ??? ?? ?? 5?? ??. 2. H.264/AVC - I Frame Interval(GOP)
¢ ?? ??? ???(Advanced Audio Coding, AAC, ISO/IEC 13818-7) ¢ ??? ????? ??? ???? ?? ??? ???? ¢ Sample rate (8~96 Khz) ¢ channel count (1, 2 ??) ¢ bitrate : Audio ?? ?? ¢ ?? : MPEG-4 Audio ?? 2. AAC
3. MediaCodec
¢ MediaCodec ¢ MediaExtractor ¢ MediaMuxer ¢ OpenGL ES ¢ NDK 3. MediaCodec
¢ Audio/Video? ???/????? API ＄ Android 4.1?? ?? ＄ Surface? ?? ??? ??? ?? ＄ Audio? AudioTrack? ???? ?? ?? ¢ Android 4.3?? ＄ Surface? ?? ??? ??.(OpenGL ES, EGL) ＄ MediaMuxer? ?? MP4 ?? ?? ?? ＄ VP8, VP9 ???(.WebM) 3. MediaCodec
¢ "video/x-vnd.on2.vp8" - VP8 video (i.e. video in .webm) ¢ "video/x-vnd.on2.vp9" - VP9 video (i.e. video in .webm) ¢ "video/avc" - H.264/AVC video ¢ "video/mp4v-es" - MPEG4 video ¢ "video/3gpp" - H.263 video ¢ "audio/3gpp" - AMR narrowband audio ¢ "audio/amr-wb" - AMR wideband audio ¢ "audio/mpeg" - MPEG1/2 audio layer III ¢ "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged in LATM!) ¢ "audio/vorbis" - vorbis audio ¢ "audio/g711-alaw" - G.711 alaw audio ¢ "audio/g711-mlaw" - G.711 ulaw audio 3. MediaCodec - ?? codec
¢ MediaCodec codec = MediaCodec.createDecoderByType(type); // OR ¢ MediaCodec codec = MediaCodec.createEncoderByType(type); ¢ codec.configure(format, ´); o public void configure (MediaFormat format, Surface surface, MediaCrypto crypto, int flags); ¢ codec.start(); 3. MediaCodec - init
¢ Video ??? o mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar); ? ??? ??? ?? : http://thdev.net/576 ¢ Audio ??? o format.setByteBuffer("csd-0", csd); ? Audio byte? ?? ??? ??? ?? : http://thdev.net/567 3. MediaCodec - MediaFormat
ByteBuffer[] inputBuffers = codec.getInputBuffers(); ByteBuffer[] outputBuffers = codec.getOutputBuffers(); 3. MediaCodec - Codec ??
int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs); if (inputBufferIndex >= 0) { // fill inputBuffers[inputBufferIndex] with valid data ... codec.queueInputBuffer(inputBufferIndex, ...); } ¢ public final void queueInputBuffer (int index, int offset, int size, long presentationTimeUs, int flags) 3. MediaCodec - Codec ??
int outputBufferIndex = codec.dequeueOutputBuffer(BufferInfo, timeoutUs); if (outputBufferIndex >= 0) { // outputBuffer is ready to be processed or rendered. codec.releaseOutputBuffer(outputBufferIndex, true/false); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) { outputBuffers = codec.getOutputBuffers(); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Subsequent data will conform to new format. MediaFormat format = codec.getOutputFormat(); 3. MediaCodec - Codec ??
codec.stop(); codec.release(); codec = null; 3. MediaCodec - Codec ??
¢ 16 ??? ??? ?. ¢ ???? ?? ???? ??? Surface? ??? ?. ¢ ??? ??? libYUV ??(NDK? ???? ??) ¢ ??? ? Surface? ???? ?? ???? ??. 3. MediaCodec - ??
4. MediaMuxer
MP4?? MPEG-4 Part 14 ?? MP4 (????? ISO/IEC 14496-14:2003)? MPEG-4? ??? ??? ????? ???? ?? ???? ???? ??? MPEG-4 ???? ?? .mp4 ??? ?? MPEG-4 ???? ????? .m4a
¢ Android 4.3 ?? ??? API ¢ .MP4 ?? ?? ?? ¢ MediaCodec ? ?? ?? ¢ ?? : MediaCodec? ???? ?? H.264? AAC RAW ???? ?? 4. MediaMuxer
¢ MediaMuxer muxer = new MediaMuxer("temp.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4); 4. MediaMuxer - init
// More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat() // or MediaExtractor.getTrackFormat(). // MediaFormat audioFormat = new MediaFormat(...); // MediaFormat videoFormat = new MediaFormat(...); if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Subsequent data will conform to new format. videoFormat = codec.getOutputFormat(); int videoTrackIndex = muxer.addTrack(videoFormat); muxer.start(); } 4. MediaMuxer - muxer add
int outputBufferIndex = codec.dequeueOutputBuffer(BufferInfo, timeoutUs); if (outputBufferIndex >= 0) { ByteBuffer outputBuffer = outputBuffers[outputBufferIndex]; muxer.writeSampleData(TrackIndex, outputBuffer, bufferInfo); codec.releaseOutputBuffer(outputBufferIndex,false); } 4. MediaMuxer - muxer write
muxer.stop(); muxer.release(); muxer = null; ¢ Muxer? stop()? ???? ??? Head? ??? ?? 4. MediaMuxer - muxer stop
¢ Muxer ??? 2GB ??? MP4 Head? ??? ??. o ?? ???? 2GB ??? ???. ?? ??? ?? ? ?. ¢ Audio/Video? ?? Muxer ?? ?? 1/2?? ?? ¢ MediaCodec?? ??? ??? ?? 4. MediaMuxer - ??
¢ MediaCodec ?? ?? : http://bigflake.com/mediacodec/ ¢ Android Example : https://android.googlesource.com/platform/cts/+/jb-mr2- release/tests/tests/media/src/android/media/cts ¢ MediaCodec Example : https://github.com/taehwandev/MediaCodecExample 5. ?? ???
Taehwan (taehwan@thdev.net) End
? ????? ?????? ??? ????? 4.0 ?? ????? ?? ?? ????. ? ????? ??? ?? ???? http://creativecommons.org/licenses/by/4.0/ ?(?) ?????.
¢ Google Drive : http://goo.gl/6pFI90 ?? ??

More Related Content

Android media codec ????

  • 2. ¢ ????? 1? 5?? ¢ ? ?? ???? ??! ?? ??? ??. o http://thdev.net ¢ ??? ?? ???! ¢ NDK ????? ?????. ???
  • 3. ? ? 1. ???/?????? 2. Codec??? a. H.264/AVC b. AAC 3. MediaCodec 4. MediaMuxer 5. ????? 6. Q & A
  • 4. MediaCodec? ???????? ??, ??? ????? API(MediaRecord)? ??? ...
  • 7. ¢ ??? o ????? ??, ??? ????? ?? ???? ?. ¢ ??? o ????? ??, ????? ?? ?????. 1. ???/??? ???
  • 9. Codec? Codec? ??? ??? ????? ??? ??, ????? ???, ?? ? ?? ? ? ?? ????? ????? ????? ?? ??? ?? ??, ??? ??? ??? ???? ?? - ???? : http://ko.wikipedia.org/wiki/%EC%BD%94%EB%8D%B1
  • 11. ¢ ITU-T and MPEG? JVT ?? ??? ???? ¢ ITU-T : H.264 ¢ MPEG : MPEG-4 part 10/AVC(Advanced Video Coding) 2. H.264/AVC
  • 12. ¢ ???? Bit-Rate ?? 50% ??? ??? ??? ?? ¢ ?? ?? ?? ¢ ?? ???? ???? ?? 2. H.264/AVC - ??
  • 13. ?Profile : ??? ??? ??? ? ? ¢ Baseline : ??? ???? ?? ¢ Main : ?? ? ??? ¢ Extended : ??? ????? ?? 2. H.264/AVC - Profile
  • 14. ¢ I-Frame(Intra-frame) ¢ P-Frame(Predictive-frame) ¢ B-Frame(Bi-directionally-frame) 2. H.264/AVC - Frame
  • 15. ¢ I-Frame(Intra-frame) o ?? ??? ?? ????? Main? ?? 2. H.264/AVC - Frame
  • 16. ¢ P-Frame(Predictive-frame) o ?? ???? ?? ????? ?? ? ?? ??? ???. ¢ B-Frame(Bi-directionally-frame) o ???/??? ??? ??? ??? ?? ??. o ???? ???? ?? ??. 2. H.264/AVC - Frame
  • 17. ¢ H.264? YUV420 ???? ??. o YUV? ?? ??, ????? ???? ??. o YUV? ???? ??? ?? TV?? ??? ??. 2. H.264/AVC - ???
  • 18. ¢ Android? 2?? YUV ?? ? ??. o I420, NV12 o ??? : width * height * 3 / 2 o ?? : YUV ??? ?? 2. H.264/AVC - YUV ???
  • 19. ¢ ?? ?? ???? ?? bit ? (bit/sec) ¢ ? bitrate? ????? ???. 2. H.264/AVC - Bitrate
  • 20. ¢ FPS(Frame per second) o 1?? ????? ? ??? ?? ???. ¢ MediaCodec - Frame rate 2. H.264/AVC - Frame rate(FPS)
  • 21. I-Frame ?? ??(GOP : Group Of Picture) ¢ I-Frame? I-Frame ??? ?? o I P P P P I : 5 ¢ MediaCodec - I Frame Interval o ? ??? ??? ?? ?? 5?? ??. 2. H.264/AVC - I Frame Interval(GOP)
  • 22. ¢ ?? ??? ???(Advanced Audio Coding, AAC, ISO/IEC 13818-7) ¢ ??? ????? ??? ???? ?? ??? ???? ¢ Sample rate (8~96 Khz) ¢ channel count (1, 2 ??) ¢ bitrate : Audio ?? ?? ¢ ?? : MPEG-4 Audio ?? 2. AAC
  • 24. ¢ MediaCodec ¢ MediaExtractor ¢ MediaMuxer ¢ OpenGL ES ¢ NDK 3. MediaCodec
  • 25. ¢ Audio/Video? ???/????? API ＄ Android 4.1?? ?? ＄ Surface? ?? ??? ??? ?? ＄ Audio? AudioTrack? ???? ?? ?? ¢ Android 4.3?? ＄ Surface? ?? ??? ??.(OpenGL ES, EGL) ＄ MediaMuxer? ?? MP4 ?? ?? ?? ＄ VP8, VP9 ???(.WebM) 3. MediaCodec
  • 26. ¢ "video/x-vnd.on2.vp8" - VP8 video (i.e. video in .webm) ¢ "video/x-vnd.on2.vp9" - VP9 video (i.e. video in .webm) ¢ "video/avc" - H.264/AVC video ¢ "video/mp4v-es" - MPEG4 video ¢ "video/3gpp" - H.263 video ¢ "audio/3gpp" - AMR narrowband audio ¢ "audio/amr-wb" - AMR wideband audio ¢ "audio/mpeg" - MPEG1/2 audio layer III ¢ "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged in LATM!) ¢ "audio/vorbis" - vorbis audio ¢ "audio/g711-alaw" - G.711 alaw audio ¢ "audio/g711-mlaw" - G.711 ulaw audio 3. MediaCodec - ?? codec
  • 27. ¢ MediaCodec codec = MediaCodec.createDecoderByType(type); // OR ¢ MediaCodec codec = MediaCodec.createEncoderByType(type); ¢ codec.configure(format, ´); o public void configure (MediaFormat format, Surface surface, MediaCrypto crypto, int flags); ¢ codec.start(); 3. MediaCodec - init
  • 28. ¢ Video ??? o mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar); ? ??? ??? ?? : http://thdev.net/576 ¢ Audio ??? o format.setByteBuffer("csd-0", csd); ? Audio byte? ?? ??? ??? ?? : http://thdev.net/567 3. MediaCodec - MediaFormat
  • 29. ByteBuffer[] inputBuffers = codec.getInputBuffers(); ByteBuffer[] outputBuffers = codec.getOutputBuffers(); 3. MediaCodec - Codec ??
  • 30. int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs); if (inputBufferIndex >= 0) { // fill inputBuffers[inputBufferIndex] with valid data ... codec.queueInputBuffer(inputBufferIndex, ...); } ¢ public final void queueInputBuffer (int index, int offset, int size, long presentationTimeUs, int flags) 3. MediaCodec - Codec ??
  • 31. int outputBufferIndex = codec.dequeueOutputBuffer(BufferInfo, timeoutUs); if (outputBufferIndex >= 0) { // outputBuffer is ready to be processed or rendered. codec.releaseOutputBuffer(outputBufferIndex, true/false); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) { outputBuffers = codec.getOutputBuffers(); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Subsequent data will conform to new format. MediaFormat format = codec.getOutputFormat(); 3. MediaCodec - Codec ??
  • 33. ¢ 16 ??? ??? ?. ¢ ???? ?? ???? ??? Surface? ??? ?. ¢ ??? ??? libYUV ??(NDK? ???? ??) ¢ ??? ? Surface? ???? ?? ???? ??. 3. MediaCodec - ??
  • 35. MP4?? MPEG-4 Part 14 ?? MP4 (????? ISO/IEC 14496-14:2003)? MPEG-4? ??? ??? ????? ???? ?? ???? ???? ??? MPEG-4 ???? ?? .mp4 ??? ?? MPEG-4 ???? ????? .m4a
  • 36. ¢ Android 4.3 ?? ??? API ¢ .MP4 ?? ?? ?? ¢ MediaCodec ? ?? ?? ¢ ?? : MediaCodec? ???? ?? H.264? AAC RAW ???? ?? 4. MediaMuxer
  • 37. ¢ MediaMuxer muxer = new MediaMuxer("temp.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4); 4. MediaMuxer - init
  • 38. // More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat() // or MediaExtractor.getTrackFormat(). // MediaFormat audioFormat = new MediaFormat(...); // MediaFormat videoFormat = new MediaFormat(...); if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Subsequent data will conform to new format. videoFormat = codec.getOutputFormat(); int videoTrackIndex = muxer.addTrack(videoFormat); muxer.start(); } 4. MediaMuxer - muxer add
  • 39. int outputBufferIndex = codec.dequeueOutputBuffer(BufferInfo, timeoutUs); if (outputBufferIndex >= 0) { ByteBuffer outputBuffer = outputBuffers[outputBufferIndex]; muxer.writeSampleData(TrackIndex, outputBuffer, bufferInfo); codec.releaseOutputBuffer(outputBufferIndex,false); } 4. MediaMuxer - muxer write
  • 40. muxer.stop(); muxer.release(); muxer = null; ¢ Muxer? stop()? ???? ??? Head? ??? ?? 4. MediaMuxer - muxer stop
  • 41. ¢ Muxer ??? 2GB ??? MP4 Head? ??? ??. o ?? ???? 2GB ??? ???. ?? ??? ?? ? ?. ¢ Audio/Video? ?? Muxer ?? ?? 1/2?? ?? ¢ MediaCodec?? ??? ??? ?? 4. MediaMuxer - ??
  • 42. ¢ MediaCodec ?? ?? : http://bigflake.com/mediacodec/ ¢ Android Example : https://android.googlesource.com/platform/cts/+/jb-mr2- release/tests/tests/media/src/android/media/cts ¢ MediaCodec Example : https://github.com/taehwandev/MediaCodecExample 5. ?? ???
  • 44. ? ????? ?????? ??? ????? 4.0 ?? ????? ?? ?? ????. ? ????? ??? ?? ???? http://creativecommons.org/licenses/by/4.0/ ?(?) ?????.
  • 45. ¢ Google Drive : http://goo.gl/6pFI90 ?? ??

Editor's Notes

  • #3: ????? MediaCodec ???? ???? ? ??????. ?? ? ?? ???? ??!?? ???? ??????. 1? 5??? ??? ??? ??, ?? FFMpeg? MediaCodec? ??? ?? ?????. ?? ???? ????? ??? ?? ?????.
  • #4: ?? ?? ??? ???/???? ?? ???, ????? MP4?? ?? ????? ???? ?? ? H.264, AAC? ????? ????, ?? ??? ??? Android API MediaCodec? ??? ??????.
  • #5: ???????? ??, ??? ????? API? ??? ?? ???? ? ??? ????? MediaCodec? API? ??? ??? ???? ?? ??? ???. ???? ???? ??, ???? ??? ??? ????, ???? ???? ??? ????? ??? ????? ????, ??? ????? ??????? ?? byte ?? ??? ??? MediaCodec API ???.
  • #6: ???? ???? ??? ????? ???? ?? ?????? ???. ?? ???? ????? ???? ??? ???. ???? ???? ?? ?? ???? ???? ???? ???! ?? ? ??? ??????.
  • #7: ???? ??? ???? ? ?????
  • #8: ?????? ????? ??????. ??? ?? ????? ??? ???? ?? ????. ??? ????? ??? ????, ??? ???? ?? ????.
  • #9: ??? ?? ?? ???? ???? ?? ??? ??? ???????. ???? 1? 1?? ??? ????? ??? ???, ???? ??? 2?? ?? ????? ????.
  • #10: ???? ??? ??? ????? ??? ??, ???? ???, ?? ? ?? ? ? ?? ????? ?????? ??? ????. ????? ????? ?? ??, ?? ? ??? ??? ???? ??? ????.
  • #11: ??? Android?? ?? ???? H.264? AAC? ???????.
  • #12: ITU-T? VCEG(Video Coding Expert Group)? ISO/IEC? MPEG(Moving Picture Expert Group)? JVT(Joint Video Team)?? ??? ????? ITU-T??? H.264?? ??? ????, MPEG ??? MPEG-4 part 10/AVC?? ??? ?????. ????? H.264?? ??? ? ?? ??? ????.
  • #13: 1) H.263? ??? H.264? ???? Bit-Rates?? 50% ??? ?? ??? ???? ???. ?? ?? H.265? VP9 ?? H.264? ??? 50% ??? ?? ??? ????? ???. 2) ???? ?? ?? 3) ?? ?? ?? 4) ???? ??? ??? ?????.
  • #15: ???? ??? ???? ?????? ???. H.264?? ??? ?? 3??? Frame? ????. ??? ???? ??? ???? ???? ????????.
  • #16: I-Frame? Intra-frame?? ?? ????? ??? ???. ?? ??????? ?? ???? ???, ?? ???? B, P-Frame? ?????? ???? ???? ??? ???.
  • #17: P-Frame? Predictive-frame?? ?? ????? ?? ?? ??? ???. 3?? ??? ??? ??? ? ??? ???? ?? ?? ? ????. ?? ???? P-Frame??? ?????. B-Frame? Bi-directionally-frame?? ???? ?? ??? ????? ??? ?? ??? ??? ???? ?? ???? ???. ???? ?? ???? Baseline ??????? ???? ?? ??????.
  • #18: ?? ?????? ???? ? Frame? ???? ???? ?????. H.264? YUV420 ???? ?????. YUV? ?? ??, ????? ???? ????? ???. YUV? ?? TV? ???? ??? ???? ??? ???? ?????. ?? ?? TV? ?? TV? ?? ???? ??? ??????? YUV ???? ???? ?????.
  • #19: Android??? I420? NV12 2?? ????? ???? ?? ???, I420, NV12? ???? ??x??x3/2? ???? ?? 2?? ???? ?? ???? ???. UV? ???? ??? ??? ???? ???.
  • #20: ??? ???? ???? ??? ??? ??? MediaCodec? ????? ???? ? ??? ???????. Bitrate? ?? ?? ???? ???? ??? ?? ????. H.264??? ? bitrate? ????? ??? ???.
  • #21: FPS?? ?? Frame Per Second ? ??? 1??? ???? ??? ?? ????. ?? ?? 30 FPS?? ?? ? 30?? ???? ????? ?? ??? ???. MediaCodec??? Frame rate? ???? ?? FPS? ???? ?? FPS? ???? ???? ?????.
  • #22: I-Frame ?? ??? GOP????, ?? ???? ??? ??? I-Frame?? I-Frame ??? ??? ?? ???? 5? ???. ??? 5 frame ?? ?? I-Frame? ?????? ??? ???. 5??? ?? ??? ???? ??? ???? ??? ?? ? ????. ???????? ? I-Frame ??? ?? ? ??? ?????? 1? ?? 5?? ????? ???. FPS? ??????? 1????? 30?? GOP? ?? ?? ????.
  • #23: ??? ??? AAC ???. AAC? Advanced Audio Coding? ??? ??? ???? ???? ???? ISO/IEC? ????? ?? ?????. ??? ????? ??? ???? ????? ?? ??? ????. MP3 ?? ?? ???? 8~96 Khz? ??????, ?? 48??? ???????. Android??? 1, 2?? ??? ???? ???, ?? ?????? ??? ??? ??? ????? ????? ???. AAC? ???? http://wiki.multimedia.cx/index.php?title=MPEG-4_Audio ???? ????? ???? ??? ?? ? ????.
  • #24: ???????? ??, ??? ????? API? ??? ?? ???? ? ??? ????? MediaCodec? API? ??? ??? ???? ?? ??? ???. ???? ???? ??, ???? ??? ??? ????, ???? ???? ??? ????? ??? ????? ????, ??? ????? ??????? ?? byte ?? ??? ??? MediaCodec API ???.
  • #25: MediaCodec? ???? ?? API? ??? ???. MediaCodec? ???? ???? ?????? MediaExtractor? ????? ???. ?????? MediaMuxer? ??? ?????. ?? OpenGL ES? NDK ??? ????? ? ?? ??? ?/???? ?????.
  • #26: ?? MediaCodec? ???? ???/??? ?? ??? ???? ?? ? ???? ??? ?????. ?? Android 4.1?? ?????, Surface? ?? ???? ?????. ???? byte ??? ?? ??? ?? ????? ???? API ???. Android 4.3?? ??? ??? ???? Surface? ?? ???? ????, MediaMuxer? ??? ? ?? ?????. VP8, VP9 ??? ??? ???????.
  • #28: MediaCodec? ???? ???? Decoder? Encoder? ??? ???? ???. Type? ?? ??? ?? ??? ???, ? ??? ?? Decoder? Encoder? ???? ???? ??? ???. ? ? ?? ??? Github? ???? ??????? ?? ????? ?????. ?? ??? ???? configure? ???? ???. Configure? ???/???? ??? ??? ????. ?? ??? ?????? configure? ??? ??? ?????? ?? ?? ?? ????. MediaFormat? Surface, ??? ???? ???? ????. ??? flags? ???? ???? ??? ???? ??? Encoder? 1? ???. ??? ??? start()? ???? MediaCodec ?? ??? ?? ??????.
  • #29: MediaFormat ??? ???/???? ??? ??? ?? ????. ? ??? ? ????? ??, ? ? ??? ?? 2?? ??? ???????. ???? ???? ???? ???? ???? ???. ?? ???? ?? I420, NV12? ????? ???? ??? ?? ??? ??? ??? 5?? ??? ???. ?? ???? ???? ??? ???? ??????. ?? ??? ????? ????? ?????. Audio ? byte? ??? ?? ???? ??? csd-0 ?? ??? ??? ???. MediaExtractor? ???? ???? ????? byte? ?? ????? ? ???? ???????. 2 byte? ??? ????? ??? ???? ??, samplerate, audioProfile ?????. ???? ??? ???? ?????? ????? ?????.
  • #30: ???? start? ?????? InputBuffer? outputBuffer? ???? ???. ? buffer? framework?? ???? ???? ??? ????? ? ? ????.
  • #31: ??? Buffer? ???? ???? ????, ???? ?? ????. ?? ?? ?? ??? ??? ????. dequeueInputBuffer? ?? index? ????, ?? buffer? ???? ???? queueInputBuffer? ???? ?? ???. ??? Muxer? ??? ?? public final void queueInputBuffer (int index, int offset, int size, long presentationTimeUs, int flags) ? 4?? ??? long? ????? ???. ??? ???? ?? ????? ???.
  • #32: OutputBuffer? ???? ?????. inputbuffer? ?? ??? ?? ???? ???? ???? ???. ??? ???? outputBuffer? bufferInfo? ?? ?? ?/??? ? byte? ?????. ?????? Surface? ?? ??? ??? ????? ??? ???? ???, releaseOutputBuffer()?outputIndex? boolean ?? ?? ?? Surface? Rendering? ? ? ????. ? ??? outputBufferIndex? 0 ?? ?? ???? ???? ???, ?? ????? BufferChanged? OutputFormat changed? ?? ?????.
  • #33: ?? ??? ?? ??? ???? MediaCodec ??? ??? ?? ???. ?? ??? ??? ??? ? ?? ??? ???? MediaCodec ???? ????????.
  • #34: ?? ?? ??? ??? ???? ???? MediaMuxer? ????? ?????. Video ???? ???? ???? ? ??? ?????. ?? H.264? ??? 16? ???? ???. 1920x1080? 16? ??? 1920x1088 ???. MediaCodec? ?? ?? ???? 1920x1080?? ???? ?? ?? ????. ??? ??? ?????? ???? ?? ???? ?? ? ??? 16? ??? ? ?????? ???. ???? ?? ????? ???????? Surface? ???? ?????? ????. ????? OpenGL ES? ??? ??? ?????. ??? ??? Surface? ???? ?? ???? ?????.
  • #37: MediaMuxer? ????? 4.3 ???? ???????. MediaCodec? ?? ???? ??, MP4 ?? ??? ?????. MediaCodec? ?? ?? ????? H.264? AAC? ??? ??, ?? ?????? ??? ??????. ?? MP4?? ?? ????? ????? ?? ??? ?????.
  • #39: MediaCodec? return ? ? MediaCodec.INFO_OUTPUT_FORMAT_CHANGED ?? ??? ???? ???. ? ??? Audio, Video? ???? ??? Audio? track ??? Video? track ??? return ???.
  • #40: Muxer? ??? ??, track??? return ???? ?? byte? ??? ???. MediaCodec? outputIndex? 0?? ? ???? ?? ??? ?? ????? ???. byte? BufferInfo? ????, ?? Muxer? ????? ?? ???.
  • #42: ????? Muxer? ???????. Muxer? ??? ?? ????? ??? ????. - MediaMuxer? ???? ??? ?? 2GB? ???? ???. - ???? ??? ???? Audio? 1?? 2??? ????? ??. - Muxer? ??? ?? Video/Audio? ??? ??? ????? ??.