What is an HDR image?
The Dynamic Range of real-world scenes can be quite high -- ratios of 100,000:1 are common in the natural world. An HDR (High Dynamic Range) image stores pixel values that span the whole tonal range of real-world scenes. Therefore, an HDR image is encoded in a format that allows the largest range of values, e.g. floating-point values stored with 32 bits per color channel.
Another characteristics of an HDR image is that it stores linear values. This means that the value of a pixel from an HDR image is proportional to the amount of light measured by the camera. In this sense, HDR images are scene-referred, representing the original light values captured for the scene.
Whether an image may be considered High or Low Dynamic Range depends on several factors. Most often, the distinction is made depending on the number of bits per color channel that the digitized image can hold. However, the number of bits itself may be a misleading indication of the real dynamic range that the image reproduces -- converting a Low Dynamic Range image to a higher bit depth does not change its dynamic range, of course.

· 8-bit images (i.e. 24 bits per pixel for a color image) are considered Low Dynamic Range.
· 16-bit images (i.e. 48 bits per pixel for a color image) resulting from RAW conversion are still considered Low Dynamic Range, even though the range of values they can encode is much higher than for 8-bit images (65536 versus 256). Converting a RAW file involves applying a tonal curve that compresses the dynamic range of the RAW data so that the converted image shows correctly on low dynamic range monitors. The need to adapt the output image file to the dynamic range of the display is the factor that dictates how much the dynamic range is compressed, not the output bit-depth. By using 16 instead of 8 bits, you will gain precision but you will not gain dynamic range.
· 32-bit images (i.e. 96 bits per pixel for a color image) are considered High Dynamic Range. Unlike 8- and 16-bit images which can take a finite number of values, 32-bit images are coded using floating point numbers, which means the values they can take is unlimited. It is important to note, though, that storing an image in a 32-bit HDR format is a necessary condition for an HDR image but not a sufficient one. When an image comes from a single capture with a standard camera, it will remain a Low Dynamic Range image, regardless of the format used to store it.
There are various formats available to store HDR images, such as Radiance RGBE (.hdr) and OpenEXR (.exr) among the most commonly used. See Greg Ward's HDR Image Encodings page for an excellent overview of HDR formats.

But aren't we confusing Dynamic Range with bit depth here?
Good question. Bit depth and dynamic range are indeed separate concepts and there is no direct one to one relationship between them.
The bit depth of a capturing or displaying device gives you an indication of its dynamic range capacity, i.e. the highest dynamic range that the device would be capable of reproducing if all other constraints are eliminated. For instance, a bit-depth of 12 for a CCD tells you that the maximum dynamic range of the sensor is 4096:1, but the captured dynamic range is likely to be much less once noise is taken into account (most 12-bit sensors have on average a dynamic range around 1,000:1 only).
In the case of an image file, the bit-depth in itself does not tell much about the dynamic range captured or reproduced by the file.
First, the bit depth of an image file is not a reliable indicator of the dynamic range of a reproducing device. For instance, when a RAW file is converted into a 16-bit TIFF file in linear space, the real bit-depth -and thus maximum dynamic range- of the captured data is most probably 12-bit only, which is the bit-depth of standard digital cameras. It is just because 12 bits are not convenient for computers that the file will be stored in 16 bits, but of course it does not change the dynamic range of the information stored.
Second, the bit-depth of an image file is even less a reliable indicator of the dynamic range of the scene reproduced. When a 32-bit HDR image has been properly tone mapped, it will show the original dynamic range captured, even when it is saved in an 8-bit image format. This is why a tone mapped image is often confused with an HDR image. A tone mapped image is not an HDR image as it does not represent the original values of light captured anymore. It just reproduces the dynamic range captured on standard monitors or prints.