FLV视频封装格式详解
简介：
FLV（Flash Video）是现在非常流行的流媒体格式，由于其视频文件体积轻巧、封装播放简单等特点，使其很适合在网络上进行应用，目前主流的视频网站无一例外地使用了FLV格式。另外由于当前浏览器与Flash Player紧密的结合，使得网页播放FLV视频轻而易举，也是FLV流行的原因之一。可以通过Adobe公司的RTMP（Real Time Messaging Protocol）协议连接到Adobe Flash Media Server上面发布媒体信息。FLV视频格式是Adobe公司设计开发的，目前已经免费开放，现在的版本是v10。下面我们就了解一下FLV文件格式。
FLV是流媒体封装格式，我们可以将其数据看为二进制字节流。总体上看，FLV包括文件头（Flv Header）和文件体（Flv Body）两部分，其中文件体由一系列的Tag及Tag Size对组成。Tag又可以分成三类:audio,video,script，分别代表音频流，视频流，脚本流（关键字或者文件信息之类）。
一、FLV Header
一般比较简单，包括文件类型之类的全局信息，如图：
FLV Header 一共九个字节，含义如下：
第1-3字节为文件标识（Signature），总为“FLV”（0x46 0x4C 0x56），第4字节为版本，目前为1（0x01）。第5个字节的前5位保留必须为0,第6位表示是否存在音频Tag,第7位保留必须为0,第8位表示是否存在视频Tag。第6-9个字节为UI32类型的值，表示从File Header开始到File Body开始的字节数，版本1中总为9。
FLV Header包含9个字节，含义如下表所示：
| FLV Header Field | 描述 |
| 文件类型(3byte) | 'F' (0x46) 'L' (0x4C) 'V' (0x56) |
| 版本(1byte) | File Version(For example 0x01 for FLV version 1) |
| 流信息(1byte) | UB[7]~UB[3]、UB[2]=1 Audio,UB[1]总为0，UB[0] = 1 Video |
| Header长度(4bytes) | 整个文件头的长度，一般是9（3+1+1+4），有时候后面还有些别的信息，就不是9了 |
二 、The FLV File Body
After the FLV Header，the remainder of an FLV file consists of alternating back-pointers and tags. They interleave as shown in the following table：
| FLV File Body Field | Type | Comment |
| PreviousTagSize0 | UI32 | Always 0 |
| Tag1 | FLVTAG | First tag |
| PreviousTagSize1 | UI32 | Size of previous tag, including its header. For FLV version 1,this value is 11 plus the DataSize of the previous tag. |
| Tag2 | FLVTAG | Second tag |
| … | … | … |
| PreviousTagSizeN-1 | UI32 | Size of second-to-last tag |
| TagN | FLVTAG | Last tag |
| PreviousTagSizeN | UI32 | Size of last tag |
FLV TagHeader have the following format：
| FLVTAG Field | Type | Comment |
| TagType | UI8 | Type of this tag. Values are： 8:audio 9:video 18:script data |
| DataSize | UI24 | Length of the data in the Data field |
| Timestamp | UI24 | Time in milliseconds at which the data in the this tag applies. The valve is relative to the first tag in the FLV file, which always has a timestamp of 0. |
| TimestampExtended | UI8 | Extension of Timestamp field to form a UI32 value. This field represents the upper 8 bits, while the previous Timestamp filed represents the lower 24 bits of the time in milliseconds. |
| StreamID | UI24 | Always 0 |
In playback, the time sequencing of FLV tags depends on the FLV timestamps only. Any timing mechanisms built into the playload data format are ignored.
FLV TagData的说明：
如果TagType是8, TagDate数据为音频；如果TagType是9, TagDate数据为视频；TagType是18, TagDate数据为SCRIPTDATAOBJECT.
FLV body就是由很多tag组成的。FLV文件里面帧的实体就是tag了。每个tag都可以分为两部分，第一部分包含是tag 类型信息，长度固定为15字节（PreviousTagSize + The Size of FLV TagHeader ），如图：
第二部分为tag data，也就是flv的数据(有音频，视频，脚本等三类数据)，根据不同的tag类型就有不同的数据区，数据区的长度由第一部分的数据区长度字段定义，如图：
接下来就是下一个tag的内容，其开始的四个字节定义了上个tag的总长度，注意上个tag的总长度中不包括上个tag之前的4个描述再上一个tag的长度的4个字节，如图：
接下来说一下文件尾，在文件尾的最后有四个字节是定义最后一个tag的长度的，如图：
这里我门可以算一下，是00 00 00 DD是221，最后一个tag的长度是221，如图：
FLV视频封装格式详解（二）
Overview
Flash Video(简称FLV),是一种流行的网络格式。目前国内外大部分视频分享网站都是采用的这种格式.
File Structure
从整个文件上开看,FLV是由The FLV header和The FLV File Body组成.
1.The FLV header
| Field | Type | Comment |
| Signature | UI8 | Signature byte always 'F' (0x46) |
| Signature | UI8 | Signature byte always 'L' (0x4C) |
| Signature | UI8 | Signature byte always 'V' (0x56) |
| Version | UI8 | File version (for example, 0x01 for FLV version 1) |
| TypeFlagsReserved | UB [5] | Shall be 0 |
| TypeFlagsAudio | UB [1] | 1 = Audio tags are present |
| TypeFlagsReserved | UB [1] | Shall be 0 |
| TypeFlagsVideo | UB [1] | 1 = Video tags are present |
| DataOffset | UI32 | The length of this header in bytes |
Signature: FLV 文件的前3个字节为固定的‘F’‘L’‘V’,用来标识这个文件是flv格式的.在做格式探测的时候，
如果发现前3个字节为“FLV”，就认为它是flv文件.
Version: 第4个字节表示flv版本号.
Flags: 第5个字节中的第0位和第2位,分别表示 video 与 audio 存在的情况.(1表示存在,0表示不存在)
DataOffset: 最后4个字节表示FLV header 长度.
2.The FLV File Body
| Field | Type | Comment |
| PreviousTagSize0 | UI32 | Always 0 |
| Tag1 | FLVTAG | First tag |
| PreviousTagSize1 | UI32 | Size of previous tag, including its header, in bytes. For FLV version1, this value is 11 plus the DataSize of the previous tag. |
| Tag2 | FLVTAG | Second tag |
| ... | ... | ... |
| PreviousTagSizeN-1 | UI32 | Size of second-to-last tag, including its header, in bytes. |
| TagN | FLVTAG | Last tag |
| PreviousTagSizeN | UI32 | Size of last tag, including its header, in bytes |
FLV header之后,就是FLV File Body.
FLV File Body是由一连串的back-pointers + tags构成.back-pointers就是4个字节数据,表示前一个tag的size.
FLV Tag Definition
FLV文件中的数据都是由一个个TAG组成,TAG里面的数据可能是video、audio、scripts.
下表是TAG的结构:
1.FLVTAG
| Field | Type | Comment |
| Reserved | UB [2] | Reserved for FMS, should be 0 |
| Filter | UB [1] | Indicates if packets are filtered. 0 = No pre-processing required. 1 = Pre-processing (such as decryption) of the packet is required before it can be rendered. Shall be 0 in unencrypted files, and 1 for encrypted tags. See Annex F. FLV Encryption for the use of filters. |
| TagType | UB [5] | Type of contents in this tag. The following types are defined: 8 = audio 9 = video 18 = script data |
| DataSize | UI24 | Length of the message. Number of bytes after StreamID to end of tag (Equal to length of the tag – 11) |
| Timestamp | UI24 | Time in milliseconds at which the data in this tag applies. This value is relative to the first tag in the FLV file, which always has a timestamp of 0. |
| TimestampExtended | UI8 | Extension of the Timestamp field to form a SI32 value. This field represents the upper 8 bits, while the previous Timestamp field represents the lower 24 bits of the time in milliseconds. |
| StreamID | UI24 | Always 0. |
| AudioTagHeader | IF TagType == 8 AudioTagHeader |
|
| VideoTagHeader | IF TagType == 9 VideoTagHeader |
|
| EncryptionHeader | IF Filter == 1 EncryptionTagHeader |
|
| FilterParams | IF Filter == 1 FilterParams |
|
| Data | IF TagType == 8 AUDIODATA IF TagType == 9 VIDEODATA IF TagType == 18 SCRIPTDATA |
Data specific for each media type. |
TagType: TAG中第1个字节中的前5位表示这个TAG中包含数据的类型,8 = audio,9 = video,18 = script data.
DataSize:StreamID之后的数据长度.
Timestamp和TimestampExtended组成了这个TAG 包数据的PTS信息,记得刚开始做FVL demux的时候，并没有考虑TimestampExtended的值,直接就把Timestamp默认为是PTS，后来发生的现 象就是画面有跳帧的现象,后来才仔细看了一下文档发现真正数据的PTS是PTS= Timestamp | TimestampExtended<<24.
StreamID之后的数据就是每种格式的情况不一样了，接下格式进行详细的介绍.
Audio Tags
如果TAG包中的TagType==8时，就表示这个TAG是audio。
StreamID之后的数据就表示是AudioTagHeader，AudioTagHeader结构如下：
| Field | Type | Comment |
| SoundFormat | UB [4] | Format of SoundData. The following values are defined: 0 = Linear PCM, platform endian 1 = ADPCM 2 = MP3 3 = Linear PCM, little endian 4 = Nellymoser 16 kHz mono 5 = Nellymoser 8 kHz mono 6 = Nellymoser 7 = G.711 A-law logarithmic PCM 8 = G.711 mu-law logarithmic PCM 9 = reserved 10 = AAC 11 = Speex 14 = MP3 8 kHz 15 = Device-specific sound Formats 7, 8, 14, and 15 are reserved. AAC is supported in Flash Player 9,0,115,0 and higher. Speex is supported in Flash Player 10 and higher. |
| SoundRate | UB [2] | Sampling rate. The following values are defined: 0 = 5.5 kHz 1 = 11 kHz 2 = 22 kHz 3 = 44 kHz |
| SoundSize | UB [1] | Size of each audio sample. This parameter only pertains to uncompressed formats. Compressed formats always decode to 16 bits internally. 0 = 8-bit samples 1 = 16-bit samples |
| SoundType | UB [1] | Mono or stereo sound 0 = Mono sound 1 = Stereo sound |
| AACPacketType | IF SoundFormat == 10 UI8 |
The following values are defined: 0 = AAC sequence header 1 = AAC raw |
AudioTagHeader的头1个字节，也就是接跟着StreamID的1个字节包含着音频类型、采样率等的基本信息.表里列的十分清楚.
AudioTagHeader之后跟着的就是AUDIODATA数据了，也就是audio payload 但是这里有个特例，如果音频格式（SoundFormat）是10 = AAC，AudioTagHeader中会多出1个字节的数据AACPacketType，这个字段来表示AACAUDIODATA的类型：0 = AAC sequence header，1 = AAC raw。
| Field | Type | Comment |
| Data | IF AACPacketType ==0 AudioSpecificConfig | The AudioSpecificConfig is defined in ISO14496-3. Note that this is not the same as the contents of the esds box from an MP4/F4V file. |
| ELSE IF AACPacketType == 1 Raw AAC frame data in UI8 [ ] | audio payload |
AAC sequence header也就是包含了AudioSpecificConfig，AudioSpecificConfig包含着一些更加详细音频的信息，AudioSpecificConfig的定义在ISO14496-3中1.6.2.1 AudioSpecificConfig，这里就不详细贴了。而且在ffmpeg中有对AudioSpecificConfig解析的函数，ff_mpeg4audio_get_config(),可以对比的看一下，理解更深刻。
AAC raw这种包含的就是音频ES流了，也就是audio payload.
在FLV的文件中，一般情况下AAC sequence header这种包只出现1次，而且是第一个audio tag，为什么要提到这种tag，因为当时在做FLVdemux的时候，如果是AAC的音频，需要在每帧AAC ES流前边添加7个字节ADST头,ADST在音频的格式中会详细解读，这是解码器通用的格式，就是AAC的纯ES流要打包成ADST格式的AAC文件，解码器才能正常播放.就是在打包ADST的时候，需要samplingFrequencyIndex这个信息，samplingFrequencyIndex最准确的信息是在AudioSpecificConfig中，所以就对AudioSpecificConfig进行解析并得到了samplingFrequencyIndex。
到这步你就完全可以把FLV 文件中的音频信息及数据提取出来，送给音频解码器正常播放了。
Video Tags
如果TAG包中的TagType==9时，就表示这个TAG是video.
StreamID之后的数据就表示是VideoTagHeader，VideoTagHeader结构如下：
| Field | Type | Comment |
| Frame Type | UB [4] | Type of video frame. The following values are defined: 1 = key frame (for AVC, a seekable frame) 2 = inter frame (for AVC, a non-seekable frame) 3 = disposable inter frame (H.263 only) 4 = generated key frame (reserved for server use only) 5 = video info/command frame |
| CodecID | UB [4] | Codec Identifier. The following values are defined: 2 = Sorenson H.263 3 = Screen video 4 = On2 VP6 5 = On2 VP6 with alpha channel 6 = Screen video version 2 7 = AVC |
| AVCPacketType | IF CodecID == 7 UI8 |
The following values are defined: 0 = AVC sequence header 1 = AVC NALU 2 = AVC end of sequence (lower level NALU sequence ender is not required or supported) |
| CompositionTime | IF CodecID == 7 SI24 |
IF AVCPacketType == 1 Composition time offset ELSE 0 See ISO 14496-12, 8.15.3 for an explanation of composition times. The offset in an FLV file is always in milliseconds. |
VideoTagHeader的头1个字节，也就是接跟着StreamID的1个字节包含着视频帧类型及视频CodecID最基本信息.表里列的十分清楚.
VideoTagHeader之后跟着的就是VIDEODATA数据了，也就是video payload.当然就像音频AAC一样，这里也有特例就是如果视频的格式是AVC（H.264）的话，VideoTagHeader会多出4个字节的信息.
AVCPacketType 和 CompositionTime。AVCPacketType 表示接下来VIDEODATA （AVCVIDEOPACKET）的内容：
IF AVCPacketType == 0AVCDecoderConfigurationRecord（AVC sequence header）
IF AVCPacketType == 1One or more NALUs (Full frames are required)
AVCDecoderConfigurationRecord.包含着是H.264解码相关比较重要的sps和pps信息，再给AVC解码器送数据 流之前一定要把sps和pps信息送出，否则的话解码器不能正常解码。而且在解码器stop之后再次start之前，如seek、快进快退状态切换等，都 需要重新送一遍sps和pps的信息.AVCDecoderConfigurationRecord在FLV文件中一般情况也是出现1次，也就是第一个 video tag.
AVCDecoderConfigurationRecord的定义在ISO 14496-15, 5.2.4.1中，这里不在详细贴，
SCRIPTDATA
如果TAG包中的TagType==18时，就表示这个TAG是SCRIPT.
SCRIPTDATA 结构十分复杂，定义了很多格式类型，每个类型对应一种结构.
| Field | Type | Comment |
| Type | UI8 | Type of the ScriptDataValue. The following types are defined: 0 = Number 1 = Boolean 2 = String 3 = Object 4 = MovieClip (reserved, not supported) 5 = Null 6 = Undefined 7 = Reference 8 = ECMA array 9 = Object end marker 10 = Strict array 11 = Date 12 = Long string |
| ScriptDataValue | IF Type == 0 DOUBLE IF Type == 1 UI8 IF Type == 2 SCRIPTDATASTRING IF Type == 3 SCRIPTDATAOBJECT IF Type == 7 UI16 IF Type == 8 SCRIPTDATAECMAARRAY IF Type == 10 SCRIPTDATASTRICTARRAY IF Type == 11 SCRIPTDATADATE IF Type == 12 SCRIPTDATALONGSTRING |
Script data value. The Boolean value is (ScriptDataValue ≠ 0). |
类型在FLV的官方文档中都有详细介绍.
onMetaData
onMetaData 是SCRIPTDATA中对我们来说十分重要的信息，结构如下表：
| Property Name | Type | Comment |
| audiocodecid | Number | Audio codec ID used in the file (see E.4.2.1 for available SoundFormat values) |
| audiodatarate | Number | Audio bit rate in kilobits per second |
| audiodelay | Number | Delay introduced by the audio codec in seconds |
| audiosamplerate | Number | Frequency at which the audio stream is replayed |
| audiosamplesize | Number | Resolution of a single audio sample |
| canSeekToEnd | Boolean | Indicating the last video frame is a key frame |
| creationdate | String | Creation date and time |
| duration | Number | Total duration of the file in seconds |
| filesize | Number | Total size of the file in bytes |
| framerate | Number | Number of frames per second |
| height | Number | Height of the video in pixels |
| stereo | Boolean | Indicating stereo audio |
| videocodecid | Number | Video codec ID used in the file (see E.4.3.1 for available CodecID values) |
| videodatarate | Number | Video bit rate in kilobits per second |
| width | Number | Width of the video in pixels |
这里面的duration、filesize、视频的width、height等这些信息对我们来说很有用.
keyframes
当时在做flv demux的时候，发现官方的文档中并没有对keyframes index做描述，但是flv的这种结构每个tag又不像TS有同步头，如果没有keyframes index 的话，seek及快进快退的效果会非常差，因为需要一个tag一个tag的顺序读取。后来通过网络查一些资料，发现了一个keyframes的信息藏在SCRIPTDATA中。
keyframes几乎是一个非官方的标准，也就是民间标准.在网上已经很 难看到flv文件格式，但是metadata里面不包含 keyframes项目的视频 . 两个常用的操作metadata的工具是flvtool2和FLVMDI，都是把keyframes作为一个默认的元信息项目.在FLVMDI的主页 (http://www.buraks.com/flvmdi/)上有描述：
keyframes: (Object) This object is added only if you specify the /k switch. 'keyframes' is known to FLVMDI and if /k switch is not specified, 'keyframes' object will be deleted.
'keyframes' object has 2 arrays: 'filepositions' and 'times'. Both arrays have the same number of elements, which is equal to the number of key frames in the FLV. Values in times array are in 'seconds'. Each correspond to the timestamp of the n'th key frame. Values in filepositions array are in 'bytes'. Each correspond to the fileposition of the nth key frame video tag (which starts with byte tag type 9).
也就是说keyframes中包含着2个内容 'filepositions' and 'times'分别指的是关键帧的文件位置和关键帧的PTS.通过keyframes可以建立起自己的Index，然后再seek和快进快退的操作中，快速有效的跳转到你想要找的关键帧的位置进行处理。
湘公网安备 43102202000103号