- FIFO File writing
- Posted by Kirsten on August 26th, 2004
Hi @all,
This night I searched the whole internet... and I cannot believe that there is no possibility the write to a file "first in -
first out"? Well, how to implement a file logging system? How to implement history file writer?
I want to add a new line at the end of my log file, after that the first line should deleted, so my file cannot get bigger and
bigger. Working in memory this data structure would be called queue. But I want/need it for files (immediate physical saving).
I don't want to write circular either, so I don't want to start at the beginning of the file again... (I found some
implementations doing this).
Why can the file system reset the end of a file (EOF), but not the beginning?
Have I forgot something? Are there any functions which can help? How can FIFO File writing being realized?
(BTW I'm working with WinXP)
Greetings,
Kirsten
- Posted by Ben Pfaff on August 26th, 2004
"Kirsten" <stefanklauer@gmx.de> writes:
The usual technique for log files is to start a new log file
every so often and delete the oldest one(s).
--
Ben Pfaff
email: blp@cs.stanford.edu
web: http://benpfaff.org
- Posted by CBFalconer on August 26th, 2004
Kirsten wrote:
First, fix your line length. I had to reformat the above. Your
lines should not exceed 78 chars ever, and 65 is much better.
Create a file of fixed length records. Reserve the zeroth record
to show the current write pointer, which will always be pointing
at the oldest record. To read the file, read record 0, seek the
appropriate record, read from there to the end, and from record 1
up to the initial record.
To write a new record, read the 0th, form the pointer, seek there
and write the new record. Then seek the 0th and write back
whatever you represent the current pointer (incremented) by.
The easiest way would be to make all those records simply strings
of ASCII chars. But you will want to put them in and out with
binary operations, i.e. this is a binary file.
The file is vulnerable between writing the new record and writing
the 0th index record, so you probably need to open it with
exclusive write access. You might be able to inhibit any readers
by first writing an invalid marker to the index record, but watch
out for recovery from crashes.
Figure out the initialization for yourself.
--
Chuck F (cbfalconer@yahoo.com) (cbfalconer@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!
- Posted by Kirsten on August 26th, 2004
Thanks for your good suggestion!
I think that the point of your anser is: "You cannot make FIFO files"!
I've thought about your answer and came to the conclusion: good, but not what I really want:
- exclusive write access prevents reading the file while debugging
- the file is hard to read (always seek to the first line and then goto the current pos)
"CBFalconer" <cbfalconer@yahoo.com> schrieb im Newsbeitrag news:412D5219.727A1DC8@yahoo.com...
- Posted by Willem on August 26th, 2004
Kirsten wrote:
) Thanks for your good suggestion!
)
) I think that the point of your anser is: "You cannot make FIFO files"!
Well, yeah.. The problem is the way that files are stored in the file
system. If you remove stuff from the beginning, the data from the file has
to be physically moved towards the beginning to fill the gap, as it were.
If you remove stuff from the end, that's not a problem. THink of a file as
a stack of papers that's sitting on your desk: easy to remove the top one,
hard to remove the bottom one.
Theoretically, if you know a lot about the filesystem itself, you might be
able to remove a filesystem-blocksize worth of data from the beginning by
changing the inode directly or something like that. But that's highly
non-portable, and hard to do.
I seem to recall that there is at least one filesystem (on a BSD
flavour IIRC) where you can set a file's maximum length and it
automatically becomes a FIFO. But I may have been dreaming that...
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
- Posted by Jim Rogers on August 26th, 2004
Willem <willem@stack.nl> wrote in message news:<slrncirckq.2ka9.willem@toad.stack.nl>...
Most Unices and Linux support FIFOs. They are often called
named pipes.
See reference
http://developers.sun.com/solaris/ar...med_pipes.html
Jim Rogers
- Posted by Jeff Kish on August 26th, 2004
On 26 Aug 2004 08:15:12 -0700, jimmaureenrogers@worldnet.att.net (Jim Rogers)
wrote:
specify the maxsize so that the oldest stuff goes away as the new data fills
up the fifo?
regards
Jeff Kish
- Posted by Willem on August 26th, 2004
Jim wrote:
) Most Unices and Linux support FIFOs. They are often called
) named pipes.
)
) See reference
) http://developers.sun.com/solaris/ar...med_pipes.html
That's not what the OP wanted. Sorry for snipping too much of the context.
To be short: he wanted a logfile that acted like a FIFO, that is, if it
gets too full, the beginning of the file is cut off.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
- Posted by Kirsten on August 26th, 2004
Named pipes...
I have a book which has a chapter (3 pages) about named pipes (Win32 API)! I have read it several times, but I don't get the
thing!!
Pipe => put something in, it comes out later on the other side...
Connection to file handles => has something to do with REAL files?
Well, I thought, that named pipes have something to do with FIFO and the file system, so I read the chapter again and again. But
there are NO examples! I don't know why named pipes exist in Windows? What's their purpose?
I googled... but I can only find named pipes in connection with linux (where bounds between file system and memory disappear
anyway)
"Jim Rogers" <jimmaureenrogers@worldnet.att.net> schrieb im Newsbeitrag news:82347202.0408260715.6cc1f849@posting.google.c om...
- Posted by Robert Wessel on August 27th, 2004
"Kirsten" <stefanklauer@gmx.de> wrote in message news:<cgl1pd$7al$00$1@news.t-online.com>...
Named pipes are used to communicate between two running programs,
potentially on different machines. Much like TCP, although Windows
named pipes are controlled by the normal Windows security mechanisms,
and are built on Windows Networking. There is no "real" file
involved, and other than some buffering during the communications
process, no actual storage of the data it transfers.
And the abstraction of all sorts of stuff to "file" handles is quite
common. Under Windows and *nix, pipes, files, serial ports, consoles,
printers, various network connections, and a host of other stuff is
model as some variation on a byte-stream "file" object, typically
represented with a handle from a (more-or-less) common name space.