Error: File seek failed

I ran in a problem with tripwire, it did not want to generate the <hostname>.twd file under /var/lib/tripwire. You know that the file is missing because you receive emails (reports) by tripwire that the file is missing.

I could see that because it reports, once a day, whether tripwire ran sucessfully or not. It was not and the main error was: the tripwire database does exist which is actually shown as:

### Error: File could not be opened.
### Filename: /var/lib/tripwire/<hostname>.twd
### No such file or directory
### Exiting...

That, in itself, looks easy to fix. So I ran tripwire --init and voià!

sudo tripwire --init

As usual, it prints a few errors and ignored them.

The next day, same error! The database file is still missing!!! I checked again and sure enough the file was not there. So I re-ran the --init command and this time I looked closer at the errors. Got one that broke the process and thus tripwire would not save anything:

Generating the database...
*** Processing Unix File System ***
### Warning: File system error.
### Filename: /var/lib/tripwire/<hostname>.twd
### No such file or directory
### Continuing...
### Warning: File system error.
### Filename: /etc/rc.boot
### No such file or directory
### Continuing...
### Error: File seek failed.
### Filename: /root/backup/some-file.tar.gz
### Success
### Exiting...

Now, as you can see tripwire is being funny here: it tells me "Success". But the truth is that the "Error: File seek failed" prevents the saving of the database. This happens because the file named

/root/backup/some-file.tar.gz

is more than 2Gb (maybe 4Gb?) For sure, it is not capable of dealing with very large files. Our file was 22.5Gb and somehow the seek command failed on tripwire.

Our solution was simply to move that file. We did not actually need that file to be checked by tripwire anyway. It just ended up "in the wrong place". After that, the database was properly getting created. By the way, when the database file is created, you get a message like this:

Wrote database file: /var/lib/tripwire/jc.twd
The database was successfully generated.

Until then, the ### Success is a lie!

Tripwire Crashing (18.04)

At times, tripwire crashes on initialization. It happened to me a while back and I did not make a note of it. It happened again when I switched to 18.04 from 16.04. I changed computers so I did not copy the files, I just setup a brand new setup.

Here are the errors that tripwire generated:

The object: "/dev/hugepages" is on a different file system...ignoring.
The object: "/dev/mqueue" is on a different file system...ignoring.
The object: "/dev/pts" is on a different file system...ignoring.
The object: "/dev/shm" is on a different file system...ignoring.

The fact is that the software tries to ignore those errors but apparently it enters a state which makes it crash before the initialization is done and as a result you do not get the <hostname>.key file which is required to then check the changes on a daily basis.

The software, at that point, must be doing something wrong such as trying to access a pointer which is still NULL. Anyway, there is an easy way to fix the problem (a work around) which is to comment out the failing directories. Actually I completely commented out the /proc and /dev directories.

sudo vim /etc/tripwire/twpol.txt

Search for the section with /dev and /proc and comment it out completely like so:

#
# Critical devices
#
#(
#  rulename = "Devices & Kernel information",
#  severity = $(SIG_HI),
#)
#{
#       /dev            -> $(Device) ;
#       #/proc          -> $(Device) ;
#}

Since we changed the twpol.txt file, we have to re-process the file as follow:

cd /etc/tripwire/
sudo twadmin --create-polfile --cfgfile tw.cfg \
                       --site-keyfile site.key twpol.txt

The command will ask you for your secret to use the key you created at the time tripwire was installed. It takes a moment and then you can attempt the initialization again:

sudo tripwire --init

This time, it shouldn't crash.

Tripwire Still Crashing (20.04)

I now have 20.04 and tripwire crashes again.

They mentioned editing the configuration file setting a variable to false:

RESOLVE_IDS_TO_NAMES=false

This worked for me.

Don't forget to run twadmin as mentioned by Philip in a comment:

twadmin --create-cfgfile \
        --site-keyfile /etc/tripwire/site.key \
        /etc/tripwire/twcfg.txt

It looks, though, that doesn't help much for everyone... Another solution (which I did not yet try) is to recompile from the source.

Tripwire Crashing in 22.04

According to a comment on launchpad about tripwire crashes in 22.04, the package comes straight from Unstable Debian instead of being recompiled for Ubuntu 22.04.

Recompiling locally from source would fixes and the user who posted that comment actually created a PPA with the working version. So you can install his version instead.

Bug report:
https://bugs.launchpad.net/ubuntu/+source/tripwire/+bug/1968430

PPA with recompiled version for Ubuntu 22.04:
https://launchpad.net/~xuzhen666/+archive/ubuntu/lts

To just get the .deb you can download it from here:
https://ppa.launchpadcontent.net/xuzhen666/lts/ubuntu/pool/main/t/tripwire/

I verified that the source was indeed exactly the same and it was.

Real "Fix"?

Once I tried on a DigitalOcean system, this failed. I installed some items for perl and recompiled a few things with cpanm. That created files with a user identifier set to 501. The fact is, there was no more user 501 (if there was one at any point in time?). At the time tripwire SEGV, it happens because the user is missing from the /etc/passwd file. So you have three solutions:

1. Try creating the missing user
2. Find all the files with that user UID and fix the user to something else (i.e. root)
3. Find all the files with that user UID and if not necessary (cpanm created those work files as intermediate files) delete them all.

How did I find that user identifier (UID)?

I ran tripwire in the debugger after recompiling it. It may be sufficient with the existing version (/usr/sbin/tripwire instead of ./tripwire).

gdb ./tripwire

Once it crashed with the SEGV, try the command to see the stack:

gdb> where

You may have to hit enter once. At some point you should see a line like so:

#4  0x0000000000481b6f in cUnixFSServices::GetUserName (this=<optimized out>, user_id=501, tstrUser=...) at ../core/unixfsservices.cpp:542

And as you can see, the "user_id" parameter is set to 501 just before the crash.

Here are a more complete stack trace:

#0  0x00007ffff7bf86a4 in _nss_systemd_is_blocked () from /lib/x86_64-linux-gnu/libnss_systemd.so.2
#1  0x00007ffff7bfa46d in _nss_systemd_getpwuid_r () from /lib/x86_64-linux-gnu/libnss_systemd.so.2
#2  0x00000000005e5bcf in getpwuid_r ()
#3  0x00000000005e59d3 in getpwuid ()
#4  0x0000000000481b6f in cUnixFSServices::GetUserName (this=<optimized out>, user_id=501, tstrUser=...) at ../core/unixfsservices.cpp:542
#5  0x0000000000459892 in cFSPropDisplayer::InitForProp (this=0x760be0, pFCO=<optimized out>, propIdx=<optimized out>)
    at ../fs/./../core/fsservices.h:353
#6  0x0000000000453ab6 in cFSPropDisplayer::InitForFCO (this=0x760be0, ifco=0x8e2550) at ../fs/fspropdisplayer.cpp:248
#7  0x0000000000427631 in cTripwireUtil::CalcProps (pFCO=0x8e2550, pSpec=<optimized out>, pCalc=<optimized out>, pPD=0x760be0)
    at ./src/tripwire/tripwireutil.cpp:79
#8  0x0000000000423740 in util_ProcessDir (dbIter=..., pIter=0x9a0110, pSpec=0x770330, pPC=0x747070, pPD=0x760be0)
    at ./src/tripwire/generatedb.cpp:92
#9  0x000000000042389f in util_ProcessDir (dbIter=..., pIter=0x99b370, pSpec=0x770330, pPC=0x747070, pPD=0x760be0)
    at ./src/tripwire/generatedb.cpp:105
#10 0x000000000042389f in util_ProcessDir (dbIter=..., pIter=0x949370, pSpec=0x770330, pPC=0x747070, pPD=0x760be0)
    at ./src/tripwire/generatedb.cpp:105
#11 0x000000000042389f in util_ProcessDir (dbIter=..., pIter=0x9482b0, pSpec=0x770330, pPC=0x747070, pPD=0x760be0)
    at ./src/tripwire/generatedb.cpp:105

And the function where the crash occurs (tls_get_add):

0x00007ffff7bf86a4 in _nss_systemd_is_blocked () from /lib/x86_64-linux-gnu/libnss_systemd.so.2
(gdb) disassemble
Dump of assembler code for function _nss_systemd_is_blocked:
   0x00007ffff7bf8690 <+0>:    endbr64
   0x00007ffff7bf8694 <+4>:    sub    $0x8,%rsp
   0x00007ffff7bf8698 <+8>:    lea    0x44919(%rip),%rdi        # 0x7ffff7c3cfb8
   0x00007ffff7bf869f <+15>:    call   0x7ffff7bf7c80 <__tls_get_addr@plt>
=> 0x00007ffff7bf86a4 <+20>:    mov    0x1c(%rax),%eax
   0x00007ffff7bf86aa <+26>:    test   %eax,%eax
   0x00007ffff7bf86ac <+28>:    setne  %al
   0x00007ffff7bf86af <+31>:    add    $0x8,%rsp
   0x00007ffff7bf86b3 <+35>:    ret    
End of assembler dump.

How did I find the files with that user identifier (UID)?

The "find" command allows you to search your disk for any type of file and run a test on each one of those files. First I created a small script because I need to get the UID and then compare against 501. I did so in this way:

#!/bin/sh
user_id=`stat --format="%u" "$1"`
if test "${user_id}" -eq "501"
then
        echo "$1 -> ${user_id}"
fi

The "stat" may fail if the file disappear by the time the script runs. This happens when you check files under "/proc". This should not be a big issue, although in my case it still filled up 3 pages with error messages.

Then I used the following find command:

sudo find / -exec ./check-uid.sh "{}" \;

Notice the "sudo". This is important because we want to make sure we find any one file anywhere on disk. Many files are protected and only root can read them all (well... good luck if you have SELinux).

The script above is rnu with every single filename. If it detects a file where the UID is set to 501, it prints its filename in your console.

Side Note

The trick of commenting our some entries seemed to work before. I'm not too sure why it did not work for me this time. Even with the whole /root folder commented out, the tripwire command would still SEGV.

Note that the SEGV happens in libc. It looks like the static thread variable in the tripwire library has a NULL pointer and that is not considered possible so the function trying to use said pointer crashes since it tries to read from a NULL pointer.

I'm not too sure how to explain that issue. tripwire itself may be just fine, but so far this is the only process I've seen failing in this way.

Building Own Version

There are instructions on Ask Ubuntu on how to build from source:

$ apt-get source package
$ sudo apt-get build-dep package
$ dch -i
$ debuild -us -uc -b
$ sudo dpkg -i ../package.deb

The dch command is used to edit the changelog. This is not a required step. You can install your newly compiled version over the existing version when directly using dpkg. It is actually a good idea not to change the version if you want to continute to receive new updates automatically (and thus know when you have to recompile a version if such arise).

The build-dep is a step that allows for downloading all the dependencies necessary to build this version of tripwire. However, that did not work correctly for me. If you look at the files from Xu Zhen, who recompiled this for 22.04, he made edits to the deian/... files. So there are incompatible things in the default source.

The tripwire code itself, though, was not modified at all.