NB: This is a page of Linuxy geekiness. I'd imagine most of it doesn't apply to other systems, except perhaps for the bit about gdb.

A tale of woe (and argh)

In working on something for Shingo recently I had a jolly little accident that lost me a day's work. Don't say "Oh you should've saved it!", because I'd been too busy struggling with what I was doing, and was actually in the process of saving it when the editor decided that actually I really wanted to throw it all away, no questions asked, no changing my mind, via the "Revert to saved" command that was in the way when the mouse decided to release the drag prematurely.

Obviously the thing to do would be "undo revert", but in this particular case the "undo" was greyed out... So instead I decided to sleep on it. When I got up, one option that struck me as feasible was checking the process memory- quite possibly it may have released the memory used, but without Libc giving it back to the kernel yet (I'm given to understand this tends to happen, to avoid memory fragmentation).

I had previously figured that gdb (the Gnu Debugger) might be able to get at this, but if the process has no references left to this hypothetical freed memory, I wouldn't have any way to find it from there. Even if it did, how would I know where to look for it? However Linux does have special files in /proc/<pid>/mem which give a representation of the memory space, so all I had to do would be read those out, and feed them to grep or something, to search for snippets of text I remember from the lost version, and if it was still there, it'd turn up. Easy! Well, you'd think so, wouldn't you?

Reading /proc/<pid>/mem

Unfortunately my vague memories of trying to look at this in the past turned out correct. If you do cat /proc/<pid>/mem it doesn't work- it tells you "no such process", cryptically- this'd be obvious if you were trying to check via the PID of something that'd expired, but it's even if it's a continuing process. No explanation.

I checked the kernel docs. No "proc" type file in there, but there was a proc.txt under the "filesystems" subdirectory, so I read that. Didn't see anything that looked relevant (looking again I see a single line mentioning that the file is "Memory held by this process", ok thanks...). Then thought to check the proc(5) manpage. This had a bit more about it, specifically:
This file can be used to access the pages of a process's memory through open(2), read(2), and fseek(3).
Well that's something. Maybe cat and friends are using the other type of open (fopen), I often forget which is which. So I threw together a little program that did the bare minimum to open one of these files on a test process, fseek to the end, and show what its file position was, just to see if this worked.

Ran it. Segfaulted. Tweaked it a bit so that it just fseeked to the start, which should be uncontroversial. Segfaulted. Made it not even try to print what the new position was, just to do the seek. Segfaulted.

Did some searches. Eventually, after finding countless pages about some kernel vulnerability that was AFAICT fixed several years ago, I found some references to "ptrace" and "PTRACE_ATTACH". It slowly transpired, though this is apparently utterly different to other Unix systems that also have /proc/<pid>/mem files, and this is not mentioned in any docs I can find anywhere, that you can only read that file if you (a)are that process itself, in which case what the hell do you need it for in the first place? You already have access to your own memory..., or (b)have used ptrace() to attach to that process, as a debugger program would.

Ok. So I looked at the ptrace() function, and the PTRACE_ATTACH mode to do that. Added such lines to my little program, just before it tried to open the file. Ran it. Segfaulted. Great.

I then also found a page claiming that you can access the file if you are the parent of the process, and give it a STOP signal. Aha! Easy! So I went to the xterm that I luckily had launched the editor from, and hit Ctrl-Z. Process stopped, and I've got the parent (the shell). Now all I have to do is cat /proc/<pid>/mem and... no such process. Ok how about cat < /proc/<pid>/mem because then the shell would be opening the file and feeding it to cat instead? No such process...

GDB to the rescue again

I started wondering about other approaches, like reading /proc/kcore but I've heard bad things about this, as it is a representation of all the memory for the entire system, and supposedly you can crash it (fixed bug? inherent feature? I forget) by simply reading it inappropriately. So... no.

(note for non-Linux users: That isn't much of a vulnerability, only the superuser (admin) can read this file at all in the first place)

What about the core file for the individual process? That's really all I want, isn't it? I think... Unfortunately getting core files is what usually happens from crashing a program, or killing it in a certain way, and normally they don't leave them on a typical system. It'd be just my luck to send it the kill signal only to find it didn't leave one and anything that might've been there was gone for good now.

Reading through the manpage for core(5), looking at the details of when they're produced and when not, I spotted something- GDB can produce core files for running processes, via the gcore command! So I connected gdb to the editor's process, checked its help for "gcore" (seemed right), and ran the command. Core dumped in the working directory, and process is still there. WOO!

Now I went to grep it. Unfortunately, not only was the text rather broken up (I suspected it might've been, but at least it seemed to be groups of complete words), there also didn't seem to be any hits for the bits of old text. It could well be, that core files don't include memory that's been freed, even if it's not been released back to the kernel. Either way, this method didn't find me my lost work, whether there's still anything to be found or not. It is however quite plausible that it'd be effective for other tasks, so the info is here for future reference.

Follow-up

It might interest you to know that having reported this issue to the project maintainers, that particular editor has now been fixed, changes appearing in the next version AFAICT. Hooray! And though that doesn't get me my work back either, I was since mostly able to roughly rewrite it anyway, remembering the general details.