When the term "compressed" is used in relation to digital files, it's
my understanding that it's basically done one of two ways - "lossless"
where the data is all there but is assembled in such a way that
there's literally a savings of space, sort of like taking a sweater
and cramming it into a small container, where the fibers of the
sweater are jammed very closely together for the purpose saving space
and very rapidly "decoded" when viewing, and "lossy" where there's
actually data removed in such a way as to having minimal impact on
what you perceive when viewing/looking at the results - and adjustable
depending on decisions regarding tradeoff of size and quality. Would
you say this is correct?

However, reflecting on this - if you're compressing something, there
must be some starting point - i.e. some standard of data that's
considered the maximum amount of data possible - which is what I
understand "uncompressed" to be. (For purposes of ths question I'm
ignoring Hi Def, since I assume it's a new/different animal.) Looking
at for example Virtualdub, it gives an option to save as uncompressed
and the files are in fact much larger than the compressed version.

So is there just one kind of "uncompressed" - just one standard? How
would you define it?

When you take say a DV file off a Digital8 camcorder, then save the
file as "uncompressed", how is that file going to be different than
say an mjpeg file you capture via a capture card and then save as
"uncompressed"?

Is there such a thing as capturing uncompressed? I've ever seen a
consumer camcorder that captured that way particularly given the
storage demands. None of my video software offers an option for
capturing uncompressed.

Thanks for all info.

"Doc" <docsavage20@yahoo.com> wrote in message
news:7cb13176-ec2d-4790-8124-31272d589ea8@s50g2000hsb.googlegroups.com...
It's really very simple. Think of it as the CURRENT codec action on your
file.

Say you have a file of RAW RGB data which has never been compressed. The
data is in pure pristine form from where ever it was generated.

Ok.. fine. That's an uncompressed image.

now. you render through a codec. Let's say it is a codec that generates a
file format which uses a losless comperession.. Like a Run Length Encoded
Targa file for instance. The file is not compressed and no information has
been lost. Maybe it has just been formateed into a different image type. but
the data has not changed.

Ok.. Fine do far.

Next you send that same series of uncompressed frames, the originals files,
through a HDV codec. Well. now you are using a lossy compression method to
compress the data into essentially a MPEG stream. could be any lossy codec..
HDV for purposes of this..

Like any other lossy codec, the result is an image with date removed to
create a smaller file size. After all that is the target. Less data.

NOW.. you take that HDV file. and you want to create an EXACT duplicate of
it. In otherwords you want to render an uncompressed version of the HDV
file.

Well.. you can either pass it through the HDV codec again and hope that
which ever app you are using is smart enough not to re-compress any data
that has not been modified..

OR you can generate a file of uncompressed frames which are extracted from
the HDV data. Perhaps a targa file again..

however, when you create these new frames they will not contain any more
information than the HDV compressed data. But they ARE uncompressed relative
to the SOURCE.

however in this case .. exact same data is in the MPEG (HDV) file in much
smaller form.

On Mar 18, 11:05*pm, "nappy" <n...@n.n> wrote:


But RAW meaning what? I get that it means there's more data, but there
has to be some way to define this type of file. Who established the
standard?

"Doc" <docsavage20@yahoo.com> wrote in message
news:49781eee-a622-43eb-97d9-6120d8909324@8g2000hse.googlegroups.com...
On Mar 18, 11:05 pm, "nappy" <n...@n.n> wrote:


But RAW meaning what? I get that it means there's more data, but there
has to be some way to define this type of file. Who established the
standard?


Think of it like this..

The device generating the image, a camera, a 3D App, Photoshop, whatever..

generates an image with a discrete numerical representation of EACH pixel.
could be 4 bits per color .. could be 8 could be 24. Doesn't matter. The
fact that this data was just born and is in its rawest, cleanest form is all
that you consider.

For many image systems it is an RGB color set. 8 bits or REd, 8 of blue and
8 of green. But there are all kinds of formats which images are originated
in. Some are in a compressed form at origination, Like a DV camera. the raw
images are compressed 5:1 before you even get to them. That is as
uncompressed as DV will ever be..


Although the term 'uncompressed' can be used to refer to a stage of process
it generally means the data is in the rawest form. RGB, YUV etc..

"Doc" wrote ...
No, there is not just one standard. It depends on how
it was generated. It also depends on how it is being stored.

It means that no form of data reduction has been
applied since the image data was first created.
"Created" could mean something that comes out
of the red, green and blue chips of a camera, or
CGI frames generated by one of nappy's render farms.

There may be no difference in the size of the data, or in
the way the information is stored. However in the case
of both DV and MJPEG (and MPEG, etc. etc.) the data
was already compressed at one point with data loss and
you can't recreate any data that was ireversably discarded.

The purpose for storing data in an uncompressed intermediate
step (for example) is to avoid any additional trips through
the lossy codec during subsequent steps. Of course, the
raw data would be compressed down to its release format
at the end of whatever process flow.

Yes, but it is not commonplace. Mainly because all
the video you have available has already been
compressed and there is no point in merely storing
it uncompressed unless there was some further
processing steps anticipated.

Correct.

And you have identified one of the major reasons.

Another reason is that most consumer cameras are
one-chip (vs. RGB 3-chip) and a form of "information
compression" is done right in the imaging chip where
color information is collected at a lower resolution
than the B/W ("Y") image.

Likely none of your sources of video is uncompressed,
so there would be little point to it.

No consumer cameras, and few industrial cameras
have uncompressed outputs. The only truly uncompressed
video out of a camera is the raw RGB that studio cameras
output. Any form of analog NTSC or PAL is compressed
right at the starting gate.

And digital forms take advantage of this analog "compression"
to save space in the digital domain. That is why (for example)
DV25 samples the Y:Pr:Pb signals at 4:1:1 in NTSC, and
4:2:0 in PAL. i.e. in NTSC it samples the color difference
signals at only 1/4 the resolution of the monochrome
Y signal.

Doc schrieb:
This is generally called "redundancy reduction" because the reason why
it got shorter was that the redundancy in the original file has been
removed.

This is called "irrelevance reduction" because data that is irrelevant
to the viewer has been removed - removed means just that, it's gone.

There isn't really a standard. It means that no attempt has been made to
remove the redundancy, or irrelevant parts of the image. For example,
a PPM image is "uncompressed" since it contains the pixel values
directly. "Uncompressed" doesn't mean that the data is still in its
original form, i.e. if I decompress a JPEG, then that's uncompressed,
but not original. One could possibly say the data is represented then
in a form that is related/close to the representation used by the
representation or reproduction device. For images, monitors represent
data as collection of RGB pixels, so uncompressed might mean here that a
red, green and blue pixel value is available for each image sample. Note
that this is different from what most cameras collect right away.
Uncompressed text means that the text is given in a form that is
directly usable by a reader (i.e. a human, in ASCII, using the terminal
as device). I don't think one can make a strict definition.

I would expect that for an uncompressed video, the elementary data
unit, namely red, green and blue pixel value per time stamp, is
represented by one data unit (i.e. a byte) in the format.

Probably, but for video, it's rarely made because video creates too much
data to really allow for the luxury of storing the data uncompressed.
Even professional equipment captures compressed (e.g. in motion JPEG2000
in the DCI standards)

So long,
Thomas

I think Doc is asking for an official ITU standard of raw video, like
"MPEG" is a standard, but it uses compression. He is asking if Full
frames mode (RGB) was ever a "standard," and if so, who established
it?

Personally, I doubt it. During the conception of MPEG, raw IMAGES were
infeasible to store on a hard disk, let alone video. Hell, it took a
while just to LOAD the goddamn low-res picture.

Well, the sweater is maybe not the best for a metaphor.

First, you are right that "compression" is basically only transform-
coding. You transform/reform a message into another representation. We
call it "compression" if the message requires less space measured in
bits.
Obviously this means that "compression" is a specific dynamic
property
of an algorithms that otherwise generally does not compress! You may
create an algorithm that you wish to compress, but as it is and must
be expressed (in programming language + system-stuff): as an generic
transform-coding algorithm, it is possible that it fails to compress
all the time, if you are using it to transform messages that resist
being "compressed".
In short, it's "let's try and see [if it compresses]".

Second, being "compressed" and "uncompressed" has only a meaning in
the
context of a specific algorithm. Let's imagine a cascade of transform-
algorithms:

message A -> transform X -> message B -> transform Y -> message C
100k -> 50k -> 25k

You see that message B is "compressed", and well C too, and well A
not ... you might understand, that maybe there are more
transformations
going on to the left and right, those that you do not see, you don't
know about. So all of the messages could really be "compressed", in
an
absolute way.
In short, every file you have on your hard-disk may really be a
"compressed" version of it's "uncompressed" representation. You don't
know! But you know, that the majority of you files is not a "ZIP-
compressed" version of "uncompressed" files, you can verify that.

So when you take a file (any file), it can be a message from
anywhere
in the transformation-cascade above.

Here "compressed" refers to "VirtualDub-compressed", and
"uncompressed"
refers to "VirtualDub-uncompressed". If those are the most smallest
files,
and the most biggest files, you don't know and is irrelevant. You
don't
want to untransform a file to it's upper-most expansion (in bits), to
call
it really and truely "uncompressed", do you?

There is no limit in expansion, so conceptually you could never ever
speak of an absolute "uncompressed" file. You can only speak of
"MPEG-uncompressed", "ZIP-uncompressed", "HDR-uncompressed",
"RGB8-uncompressed", "YCC-uncompressed", ....
Actually in the digital world everything, but all of everything, is
quantified, and thus represents/is an approximation of something more
precise. Having an 24bit RGB-picture doesn't mean it is in it's utter
most precise form of the thing it shows. If it's a picture of your
house, it's not only that you lost the 3rd dimension, you don't even
have the whole fidelity of the electromagnetic-wave spectrum captured.
Not even in 48bit, not even in 96bit. Digital means quantified.
Quantification is a transform. Everything transformed can obtain (by
accident for example) the attribute "compressed", that's true for
"lossless" and more so for "lossy".

MJPEG is lossy, so you can easily distinguish it. Otherwise, there
would not be a difference between them:

message DV -> transform -> message MJPEG -> untransform -> message DV

is the same as:

message DV -> message DV

right?

Capturing "uncompressed" is a property of the capturing-device, not
the video-editing software. My video-card offers me to capture the
SV-In as a "RGB8-uncompressed" data-stream. This depends really on
the hardware/software-device you are using.

Ciao
Niels

Doc wrote:

Nope, there are several.

Ultimately, it depends on the original capture device. (If we're
talking about natural images, that is.)

Each digital capture device has a sensor of some sort, and an A/D
converter connected to it. The sensor could be a CCD chip. It could be a
CMOS chip. It could be something that rapidly measures voltage levels on
an analog signal line (such as in a device that captures and converts
analog video signals into a digital format.)

The format of the original, captured data is tied to the physical
properties of the capture device. CCD chips, for example, often use a
Bayer filter where the sample sites for the primary colors are arranged
in a specific pattern:

<http://en.wikipedia.org/wiki/Bayer_filter>

Some digital cameras - usually the more expensive ones - allow saving
images in a so-called "RAW" format, where the pixel/color data is
arranged in that specific, hardware-dependent pattern. Cheap pocket
cameras do not usually offer this option but convert the "RAW" data
immediately to another format - and then usually to a JPEG.

There are "degrees of RAWness", if you like. For example, the raw pixel
data from a Bayer-mask equipped image sensor might not be all that
useful to an end-user since there are no similarly patterned display
devices. In order for it to _become_ useful, it will have to be
converted to some other format that better matches the hardware of the
display device.

That "other format" would most commonly be something like an even pixel
grid where each of the pixels has a specific value for its intesities of
the red, green, and blue primaries.

Since an image sensor equipped with the Bayer filter can't offer
individual red, green, and blue values for each sample site, those
values must be synthesized mathematically, by interpolating over the
"missing" portions of data.

So, in order to get more useful data out of what we started with, a
conversion had to be done.

The "normalized" RGB data is still uncompressed, just like the original
RAW image was, but some information was irrevocably lost in that
conversion, due to rounding errors and whatnot - but that's what
basically happens in almost all kinds of conversions. (Also, some
information that wasn't there in the original data was created
algorithmically.)

Further conversions could be applied to our new RGB image. For example,
we could convert the R, G, B values to Y, Cb, Cr values. The Y channel
represents luma - it's basically a grayscale version of the image - and
the Cb and Cr channels are two color-difference signals that carry the
color information.

Digital video systems (and JPEG images) usually subsample the Cb and Cr
channels, instead of retaining their information as is:

<http://en.wikipedia.org/wiki/Chroma_subsampling>

This means that less information is used for describing the color part
of the image than the luma part of the image.

This is usually done to save storage space and bandwidth. The idea is
that the human visual system does not recognize the lower resolution of
the colors as readily as it would recognize a lower overall resolution -
with the luma part of the pictures reduced as well - so we can cheat
this way a bit, if we want to be cheap. (Practically all video devices
employ this trick in one form or the other. There are different
variations and degrees to how this is done - it depends on the device
and system in question.)

So, that was yet another conversion. We still have uncompressed data,
but it's now in the Y, Cb, Cr format, and perhaps with
less-than-the-full resolution for the Cb and Cr channels.

Uncompressed, but chroma-subsampled Y, Cb, Cr data is usually the
starting point for lossy image compression algorithms, such as JPEG and
MPEG. They'll take it from there and apply their lossy compression
algorithms to that data. (When the pictures are to be viewed again, a
reverse process happens - there will be Y, Cb, and Cr data again, but
since the compression algorithm was lossy, not the exact same data that
was once put in. This will be converted to RGB for display purposes.)

In conclusion, it's basically a question of how close to the original
sensor data you want to get - and what kind of data the capture device
_allows_ you to get. Often the original, closest-to-the-hardware-sensor
data format is unavailable to the end-user... as it is immediately
converted to some other format that is a bit more useful and
"presentable". But you can still perhaps get your hands on some
intermediate uncompressed format, without having to accept data that
would have been lossily compressed. (Or then perhaps not as some devices
are just built and programmed in a such way that they only let you
access the data in some lossily-compressed format. It depends on the
device in question.)

Some analog tv cards can capture in an uncompressed Y, Cb, Cr format.
That requires lots of disk space and speedy drives, though - or striped
drive sets, for getting that I/O edge. Or at least it used to do.

--
znark

On Mar 19, 4:17 am, Industrial One <industrial_...@hotmail.com> wrote:
Some standards exist for "raw" video; there are standards for the
arrangement and values for things like YUV 4:2:2 for example. The
exact standards escape me at the moment, sorry.

"Doc" wrote...
Perhaps if you explain WHY you are asking the question,
we might have a better chance at answering it to your
satisfaction.

If there were "more data" then it wouldn't be "uncompressed".
Not following your logic here?

"Raw" isn't a "type of file" it is a state of being.
In much the same way that "raw" doesn't describe
a specific type of food, but the fact that the food
is not cooked. Whether it will never be cooked,
or whether it is just waiting to be cooked, etc.

There are codecs for "lossless compressed" and likely
even for "uncompressed" Is that what you are asking?

Codecs generally define how data is actually stored
(and recovered) in a practical file. For example there
are hundreds of codecs indexed in this website....
http://www.fourcc.org/

The "four character code" is the part at the begining of
most kinds of container files (like AVI, MOV, etc.) which
identifies which codec was used to write the file, and
conversely, which codec you must use to decode the info
in the file.

"Jim Leonard" <MobyGamer@gmail.com> wrote in message
news:e868ce66-fa80-46d7-b07f-a9923900d161@x41g2000hsb.googlegroups.com...

CCIR601 would be a good example of standard definition.

http://en.wikipedia.org/wiki/CCIR_601

Pete