Bart Bailey <me2@privacy.net> (07-05-09 08:58:41):

This is not as easy as it sounds. Let's assume a text file. When you
first created it, it had only a single block on the drive. Some time
later, you edited it while your system was under load, such that its
contents were written to swap. By editing the file, you have made it
four blocks in size. Further you have (or some program has) copied the
file somewhere else temporarly.

Last but not least, by editing it, the file has been split into two
fragments, a one block fragment and a three blocks fragment. Later you
defragmented your partition, by which the file's contents have likely
been diffused over your entire partition.

In other words, there may be an arbitrary amount of replications on your
drive. To securely make it unrestorable, you need to delete the file,
and then wipe your entire partition, including existing content, and
additionally all partitions, which are swap partitions (under
Unix/Linux), or where swap files are or were present. (pagefile.sys in
Windows). Finally all partitions, where temporary storage is or was
present.

Don't worry. You can wipe the space used by existing files, without
destroying them. I don't know of any program, which does that, though.
But you see, to reliably destroy a single piece of information,
especially if it's saved in clear-text in a file, it takes a lot of
effort.


Regards,
Ertugrul Söylemez.


--
From the fact that this CGI program has been written in Haskell, it
follows naturally that this CGI program is perfectly secure.

Ertugrul Soeylemez wrote:



Or you have just encrypted everything.


You can also wipe almost all the space that is not used by existing files.

Hm... maybe Eraser? Just came to mind...

"Sebastian G." <seppi@seppig.de> (07-05-10 13:14:28):

Almost? Are you referring to the hidden space in modern disks, which is
used for in case of faulty blocks?


I don't know. Never had to bother, since my hard-drive is encrypted.
And if I would, then my favorite tool is `shred'. It's simple, does its
job well, and is fully suitable for me.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:


No. I'm referring to a construct called "sparse B-blocks", as used in NTFS,
ReiserFS and various other B-block based filesystems. If a file is strictly
less than 1/3 of the B-block size (which is typically 4 KB), then it's
stored in a B-block node itself rather than its own block (and being
referenced). The block then contains the file as well as pointers to other
files and other B-blocks, and usually this block can't be overridden and the
filesystem typically doesn't provide any means to let you specifically
access the file area.

Even further, when such a file is deleted, the block is still in use by the
filesystem, and if not it's still reserved for use.

The funny thing is that 'shred' has become totally useless since its naive
approach is exactly what has been driven furtile by modern filesystems.
'shred' simply tries to overwrite the file content, which might simply lead
to allocation of new blocks with the old ones just being dereferenced. Not
to mention journals on journaling filesystems, transparently compressed or
encrypted files, sparse files etc.

A modern tools would ask the filesystem driver to ask which blocks are
occupied by the file and then overwrite these blocks.

"Sebastian G." <seppi@seppig.de> (07-05-11 11:51:27):

I was suggesting that he wipes all affected partitions completely,
including the space in use. I just wanted to remark that this could be
done without losing data, i.e. existing files.

I know that ReiserFS is using this technique, but I have disabled it,
because I believe it makes things slower.


I don't shred files, but entire partitions or hard-disks. Shredding a
single file is almost equivalent anyway, because of the reasons
mentioned earlier.

So shred is (or at least appears to be) suitable for my purposes, though
I don't know whether the patterns it uses are state of the art, but the
actual problems mostly are somewhere else anyway. At last my hard-disk
doesn't contain any information I had to protect from the NSA, and even
then it would still be encrypted.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:



From your description it seemed rather like you'd delete everything and
then restore from a backup.


Huh? It makes things faster and also saves space.


That's what 'dd' is good for.


Unlikely, especially since you already mentioned the problem: Instead of
overwriting old blocks, it might just deallocate them and write to new
blocks instead.



Huh? The pattern doesn't matter at all. Where have you been the last years?

Actually this is the serious approach. Most full disk encryption systems
allow you to reencrypt existing partitions and drives without losing the
data, and then it's quite easy to omit any wiping. Another common variant is
to create a new key on every boot to encrypt the swap file, and simply
discard it on shutdown.

"Sebastian G." <seppi@seppig.de> (07-05-11 18:52:03):

Yes, just that `you' is a program and `backup' is the RAM, and we take
block-sized backups, excluding the free space of the block, then shred
the block, and finally write the backup back.


It is supposed to. Unfortunately it has had the opposite effect on my
machine. Maybe I should try again, but since I'm pretty happy with it,
I'd like better not to touch the running system.


With /dev/urandom? That's gonna take ages.


Exactly. So "securely wiping a file" is almost equivalent to "wiping
the entire storage space, where the file might have been saved at any
point in time".


Secure data erasure has never been a problem for me. But if the pattern
doesn't matter, then shred does a good job. Unlike dd with
/dev/urandom, it's pretty fast.


That's how I did it, before I've got enough RAM to not need swap anymore
at all.

BTW, I've read somewhere that even RAM contents are restorable for a few
days, after power is turned off. Does anybody know something about
this?


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:



Why not /dev/zero? Anyway, this is exactly the same as 'shred'.


dd with /dev/urandom is only limited by the PRNG (hm? RC4 outpust 200MB/s on
my machine), synchronisation with /dev/random and the entropy estimation in
/dev/random. It's much more likely that you simply hit the limit due to the
data transfer rate of your HDD, which is exactly the same as 'shred'.


You've got 4 GB of RAM? And even then I'm not fully obligated to believe you.


Why don't you read Mr. Gutmann's paper on that issue? Basically you might
have some seconds at best, and for retaining the data much longer you'll
need to deep-freeze it.

"Sebastian G." <seppi@seppig.de> (07-05-13 02:45:59):

In Linux 2.6, an SHA-based generator is used. That _is_ slow -- slower
than my hard-drive. shred on the other hand uses easy-to-compute
patterns. The first and last patterns it writes are random. Between
those, a not-so-random pattern X is written, followed by ~X (i.e. its
complement).


I've got 1.5 GB of RAM, and that's enough for my purposes.


Yes, this is where I've read that. But as far as I remember, he wrote
that the data can be restored even a few hours or days later. Well, I
guess, for most encrypted systems, rubberhose cryptanalysis is still the
most effective techinque, rather than carrying a fridge around with you.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:



So why don't you simply use /dev/zero?


Hm... seems like you're not aware of the difference between memory charge
vs. memory commitment. Heck, I just have 0.5 GB of RAM, and usually don't
need more than 200 MB. Still I need to have 2 GB of swap, because this is
what's needed to guarantee that various processes (f.e. httrack) don't run
out of RAM if they actually used all the memory they're demanding. It's not
like that actual swapping would occur.

"Sebastian G." <seppi@seppig.de> (07-05-14 14:21:36):

Because there is no point in doing so. Even if you're right in that the
pattern really doesn't matter at all, there is still no reason to prefer
/dev/zero over shred. My hard disk is far slower than both of them.

Next thing is, I don't know how my hard-disk deals with zero-writes,
when there is already a zero in its internal cache. It may well be that
it's intelligent enough not to write another zero in that case.


I know the difference, but I've seldom even had the situation, where
much more than 500 MB have been committed. Even then, Linux has a
fairly useful overcommitment policy.

The problem I have is that, even though I have more RAM than I'm going
to use ever, my hard-disk space is pretty limited.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:


There is, and I've already told: shred does a very bad job.


What a nonsense. The controller nor the cache logic cares for full-zero blocks.

"Sebastian G." <seppi@seppig.de> (07-05-14 15:54:54):

But in what way? You said, the pattern doesn't matter.


That wasn't meant to be special for zeroes. What if the cache contains
a block N with the value X and you want to write the same block X to the
same position N again? I don't know, but it would make sense to just
ignore it.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:


shred doesn't care for journaling file systems. dd simply ignores any
journal and writes to the raw partition or drive.

shred doesn't care for caches. dd only hits the kernel buffers, with
guarantees of write commitment.

And for simply overwriting files or free space, there are serious
alternatives which do the job right.



If N wasn't written to the platers yet, the same would be true for 'shred',
and a simply 'sync' would resolve the issue - and the command merging would
be justified anyway. If N was already written, then there's simply no need
to write it again.

"Sebastian G." <seppi@seppig.de> (07-05-15 04:04:13):

*Sigh*, I'm still talking about an entire partition/disk. shred does
exactly the same as dd, just with a computed pattern instead of the
input from a file/device, and it overwrites multiple times by default.


Are there any free (in terms of free speech) alternatives?


However, cache hitting is unlikely in partition/disk overwriting anyway,
since with both methods you always overwrite from the first block to the
last.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:



There's a $filesystem-tools package for about every $filesystem. There are
variants of 'rm'. For Windows, there are SDelete and Eraser.

"Sebastian G." <seppi@seppig.de> (07-05-15 15:25:46):

Unfortunately nothing for my filesystems (reiserfs-3.6 and ext2). My
distribution's package tree includes this tool SecureDelete from the THC
magazine, but they rather look like infantile `1337 h4x0rz', so I'm not
so sure whether to use it. I wasn't able to find anything other, which
appeared to be widely tested. Eraser and SDelete don't help either,
because I don't have Windows.

However, nothing to bother. All important areas of my HD are encrypted,
including my personal files and temporary storage. It would be more
practical for an attacker to trojan my operating system and intercept
the key. But who could possibly be interested in my data that much?


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.

Ertugrul Soeylemez wrote:


Hm? ReiserFS-tools contain both an unerase and a secure erase tool.
Ext3tools was kinda similar.


Tried in WINE? After all, with such clean cut code it should work quite
well, and the Filesystem IOCTLs are implemented in WINE.


At the current state of science, subverting the client would be the ONLY attack.

Who cares? I'd like to see my data secure from any threat, whether I do
imagine it or not. If you always practice good security, you'll never run
into any case where you accidentally chose too low security.

"Sebastian G." <seppi@seppig.de> (07-05-18 14:34:13):

I'm unable to find it. There is a filesystem debugger, checker, maker,
resizer and a configuration program (reiserfstune). No undeleter or
eraser.


If I'm in the case that I need to wipe out a set of files (instead of
partitions), and I'm still unable to find good erasers for Linux, then I
might. Though, good to know that it works, if all else fails.


Yes, that's my way of thinking. And now I have arrived at a point where
the effort of attacking my system is much larger than the value of my
data. The problem is that there are always things like wiretaps or
trojan horses, or even cameras. You should be safe, if your attacker is
Otto or Anna. But attackers like the NSA, the GRU or the Mossad will
still get into your system if they want.


Regards,
Ertugrul Söylemez.


--
Security is the one concept, which makes things in your life stay as
they are. Otto is a man, who is afraid of changes in his life; so
naturally he does not employ security.