Some good stuff on Video Encoding!

Couple of things to keep in mind while performing video encoding:

1. Video Codecs/formats
2. File size
3. Supported Resolution on the device
4. Bitrate
5. Aspect Ratios
6. Provider Network (Edge/3G)

AT&T Typical download speeds of 700Kbps-1.7Mbps
Typical Upload speeds of 500 Kbps-1.2 Mbps

The write up below describes the co-relation between device capabilities, codecs and Provider network. Recommended Kbps is anything between 500-700Kbps.

Apple’s Choice of Video Formats

The iPod, Apple TV and iPhone will play back videos encoded using either the MPEG-4 or H.264 codecs. These are open-standard video formats, and not in any way proprietary to Apple, but at the same time do not represent a broad portion of the video content that is currently available outside of the iTunes Store. Further, this does not represent the video standard that is used by most video recording devices, TV recording devices, or commercial DVDs. The result is that finding video content from anywhere other than the iTunes Store that is already encoded in an Apple-ready format is going to be difficult, and much of this content will therefore need to be converted.

For example, most commercial video cameras use either uncompressed Digital Video (or “DV”) or MPEG-2. Commercial DVDs also use MPEG-2 as their format. Videos downloaded from the Internet can be in any number of formats, including DivX, Windows Media Video (WMV) or QuickTime, among others.

Apple’s likely reason for these particular choices of codec is that they are an open, established standard, and they both provide a very high level of video and audio quality for a given file size. MPEG-4 has historically been very good in this regard to begin with, and the H.264 codec has only improved on the quality and file size efficiency.

As one would expect, when developing a portable video playback device, the quality-to-size ratio is very important both in terms of maximizing the amount of content that can fit on the more limited storage of a portable device, as well as maximizing the battery life of the device, as larger content can require additional processing power, thus shortening battery life. The H.264 codec appears to have been a natural fit to address both requirements, as well as providing a stable, open standard for Apple to use for their preferred video format.

Content on the iTunes Store uses the H.264 codec exclusively. Content you encode yourself can be encoded into either H.264 or MPEG-4, although H.264 will generally provide better quality for a given file-size, it also takes longer to encode.

Resolution and Bit-rate

Two other important considerations with video playback and the quality of the resulting file are the resolution and the bit-rate. The resolution simply refers to the dimension of the screen image, in terms of the number of pixels wide by the number of pixels high (ie, “640×480″) while the bit-rate refers to the amount of data that is actually encoded to make up a single second of video playback, normally expressed in either kilobits per second or megabits per second. This is the same concept as bit-rates for audio formats such as AAC and MP3.

Naturally, the higher that both of these numbers are, the better the quality of the resulting image will be. These factors both work together, however, since the resolution simply specifies the number of pixels available in a specific frame, and the bit-rate specifies how much actual information is being used to generate those pixels.

Without getting into too much technical detail, modern lossy video compression formats actually work not by encoding every single pixel of every single frame of a video, but rather simply encoding the information that changes between frames. “Reference frames” are taken at certain points, and the remaining frames are built based on what changes between each frame.

To conceptualize this, imagine watching a pro golfer make a putt:  Most of the scenery, the trees, the sky, and the golf course itself remains relatively unchanged throughout the putt, with only some motion from the golfer and of course the little white ball rolling across the green. Encoding every single frame of this would take up a lot of storage space (the equivalent of several hundred high-resolution JPEG files), when in reality very little is changing between frames. As a result, only the information that is different in each frame actually needs to be stored, and this can later be applied to the original reference frame to build a smooth video playback motion.

The bottom line is that this makes for significantly more efficient file sizes and storage of video content, and in fact is the same technology that DVDs use with the MPEG-2 format. Without MPEG-2 compression, a 90-minute DVD movie would actually occupy well over 167 GB of space, rather than fitting comfortably onto a 4.7 GB DVD.

How it directly affects the concept of bit-rates, therefore, has not to do with how much data is actually being encoded for each frame, but how much data is available to record the differences in frames. A high-resolution video with a low bit-rate will produce a more blurry image as well as visible signs of “artifacting” (distortion, blocking effects and jagged edges) on high-motion action sequences.

So what resolution and bit-rate should you use to encode your video content?  Logic would suggest that one should always use the highest settings possible, but much of this has to do with what your source content is, and the limitations of the equipment you’ll be using to watch it.

For example, if you only ever intend to watch video content on the iPod’s 320×240 screen, it would obviously be a waste of disk space and encoding time to convert these videos to any resolution higher than this.

Further, if your original source video is already at a smaller resolution, there is absolutely no point in encoding it in any higher resolution or bit-rate, as you’re not going to magically gain any resolution that wasn’t there in the first place. In our experience, consumer-grade conversion tools that promise “upconversion” of video formats are generally not worth the effort.

Likewise, even though the Apple TV supports a high-definition quality of video output (1280 x 720), if you are converting standard-definition DVDs, there is no sense in encoding them in anything beyond their original resolution.

So what are the different resolutions and formats supported by Apple’s video playback devices?  The table below provides an outline of the maximum resolutions supported, as well as the output quality of the device itself:

Naturally if you want to encode content for playback on the iPod and iPhone, you will have to stick to the lower-resolution settings of these devices—640 x 480 at 1.5 mbps. Content encoded in the higher Apple TV resolutions will not play on anything but the Apple TV.

Note that the resolutions above for the iPod and iPhone are the officially supported ones, and have been simplified in Apple’s specs to represent the more standard resolutions. More advanced users have noted that it is possible to push these resolutions and bit-rates slightly, although iTunes itself will not necessarily support the transfer of these files to the iPod or iPhone. For maximum compatibility it is therefore always best to stay within the published specifications.

A Word on Frame Rates

The “frame rate”  of a video clip refers to the number of frames, or still images, for each second of video, normally expressed in “frames-per-second”  or fps for short. Normal North American TV (NTSC) is broadcast in approximately 30 fps, while theatrical movies are normally 24 fps due to being shot on film. This means that if you’re encoding a DVD movie, chances are you’ll be looking to use 24 fps, whereas most other video content you record, including your own camera content and broadcast television content will be 30 fps. With the exception of the higher Apple TV resolution noted above, all resolutions support a maximum frame rate of 30 fps.

Bit-rate and File Size

Another important consideration when encoding videos is the size of the resulting file. This will correspond directly to the bit-rate of the file rather than the resolution (although higher resolutions require higher bit-rates), and while the H.264 and MPEG-4 codecs do a very efficient job of producing high-quality video at lower bit-rates, you’re still going to ultimately be constrained by how much space you have available on your particular device and how much content you want to be able to store.

If you’re inclined to do a bit of math, you can get an easy approximation of the file-size of a converted video simply by multiplying the bit-rate by the duration of the video, using a formula like the following:
size = ( bit-rate / 8 )  *  ( length in minutes * 60 )

The resulting size will be in either kilobytes if a kbps bitrate was used, or megabytes if a mbps bit-rate was used. So, for example, to calculate the size of a 90-minute movie being encoded at 1500 kbps, you would plug the numbers in as follows:
1500 / 8 * 90 * 60
187.5 * 5400
= 1,012,500 (KB)
or approximately 1 GB.

A simple rule-of-thumb, however, is that for standard iPod-encoded content (1.5mbps) you’ll be looking at approximately 500MB per hour of video content. Apple TV content at 3 mbps will naturally be twice that size (1GB per hour).

If storage space is of primary concern, encoding at lower bit-rates will naturally save some space, but this will come at the cost of video quality. Any bit-rate less than 1 mbps for 640×480 content will generally be too low to produce reasonably viewable TV output, for example.

Consider the Source…

Keep in mind as well when determining the best bit-rate to use to always keep the source content in mind. Older TV Shows on DVD, for example, will seldom benefit from using an extremely high bit-rate, since the original video quality was relatively low to begin with. On the other hand, recent blockbuster movies with lots of action and motion should definitely be encoding with as high a bit-rate as reasonably possible for the target device.

This is particularly relevant when considering whether to encode in a bit-rate suitable for the iPod or the Apple TV. Older TV content and analog TV recordings will rarely benefit from the higher-quality settings available for the Apple TV.

Aspect Ratios

Another very important consideration when trying to decide on the optimal format in which to encode your content is the aspect ratio of the original source content.

Aspect ratio simply refers to the ratio of the width of the image to the height of the image as shown on screen. There are three common aspect ratios in use for commercial video content today:

• 4:3 used for almost all standard-definition TV broadcast content. This is sometimes also referred to as 1.33:1:

Apple Devices and Aspect Ratios

All of this creates an additional consideration when encoding video content based on your choice of device:

• The 5G iPod has a screen that is a 4:3 aspect ratio (320 x 240) and will also only output to a TV in a 4:3 aspect ratio. Widescreen videos on the iPod or TV output will either be shown letterboxed or cropped, depending on the “Widescreen” setting on the iPod itself.
• The Apple TV on the other hand is designed to output to a 16:9 widescreen HDTV. Standard TV content will therefore be shown as pillarboxed by the Apple TV. Most high-definition TVs have a “zoom” feature that can used to crop the 4:3 output instead if this is preferred.
• The iPhone has a screen with a 1.5:1 aspect ratio (480 x 320), and no TV output capabilities at this time. 4:3 content will be pillarboxed by default, and 16:9 and 2.35:1 content will be letterboxed by default. The iPhone implements a “zoom” feature to crop the image to fill the screen in either aspect ratio.

Although the aspect ratio is going to be determined by the source content, this is an important consideration when deciding on the optimal resolution due to issues of maximum horizontal and vertical resolution.

For instance, the maximum resolution of a video encoded for iPod playback is 640 x 480, which is essentially standard-definition TV resolution in a 4:3 aspect ratio. Unfortunately, this maximum resolution means that 16:9 content gets encoded for the iPod at 640 x 360 and 2.35:1 content gets encoded at 640 x 272. Both of these are significantly below the resolution of a standard TV, and will look significantly deficient when played on an HDTV.

Therefore, while encoding standard 4:3 TV content for playback on both an iPod and Apple TV may be practical, since the maximum resolution of the source material is limited to 640 x 480 anyway, encoding movie content intended for playback on the Apple TV is a tougher decision. The Apple TV itself can display 16:9 widescreen content in full 854 x 480 resolution, taking full advantage of the original DVD quality, but this will of course limit these files to being playable only on the Apple TV.

The following table illustrates the maximum resolutions that can be played by each device for the three common aspect ratios:

Note that although the Apple TV is capable of playing 4:3 content in a much higher resolution than the iPod, most of the available 4:3 content will only be in a 640 x 480 resolution to begin with, and nothing is truly gained by increasing this resolution to anything higher.

Single-Device Viewing

Of course, it stands to reason that if you’re going to be encoding video solely for the purpose of watching on one specific portable device, and never intend to watch it anywhere else (“never” being a very long-term word), then you should encode it to the screen resolution of that device for the best possible quality and optimal file-size. The iPod and iPhone will scale down anything larger to their native screen resolution anyway, so encoding anything in a larger resolution is simply a waste of space.

Keep in mind, however, that the 5G iPod also offers the capability of outputting video content to a TV, and 320 x 240 content will look noticeably poor on anything but the smallest of TV screens. This alone may be a reason to use the higher 640 x 480 resolution, even for iPod content.

Recommended Resolutions for typical content

So, armed with all of this information, what are the recommended resolutions that should be used for the content that you’re likely to be encoding?

Ultimately, the ideal resolution will be determined by the source material in question, the desired output quality, available storage space, and of course the devices on which you’ll be watching the content. However, a brief summary of the recommended maximum resolutions for compatibility with the various devices is shown in the following table:

With the exception of standard 4:3 TV content, videos encoded in the “Apple TV” resolutions noted above will only play back on the Apple TV. On the other hand, videos encoded in the lower resolutions supported by the iPhone and iPod will also play on an Apple TV, although the more limited vertical resolutions for widescreen and CinemaScope content will produce much lower-quality images when viewed on larger-screen TVs, making the decision on whether to trade off for mobile compatibility or TV viewing quality a difficult one for some.

Note as well that when encoding standard-definition DVDs for the Apple TV, considerable space can be saved when using a tool that supports anamorphic encoding, since increasing the size of a 720 x 480 video to the larger horizontal resolutions above may require a higher bit-rate to render and thereby unnecessarily increase the file size.

Conclusion

While perhaps not the most inexpensive method, the easiest way to get video content that is playable on the iPod is still to purchase it from the iTunes Store. However, this is not always practical for all users, and certainly not even available to many iPod users outside of the U.S. As a result, the only practical way for many to get video content in an Apple-ready format and build a library is to convert your own content.

Unfortunately, video conversion can be a time-consuming task, so it is important to have a good understanding of the various issues before spending many dozens of hours converting video only to discover that it is not in the most optimal format. Further, due to the time required, it is a good idea to try and create a video collection that takes into account the platforms and devices that you may want to use in the future.