Tricks for dealing with panes in tmux

Straight to the point: here’s a list of config lines/commands that I hope will ease your life as they have done with mine!

Write on all open panes at the same time

CTRL+B : to enter the tmux command line and setw synchronize-panes on.
Of course, set it “off” to disable sync’d writing.

Change panes layout

CTRL+Space to cycle through all layouts
or
CTRL+B META+15 to select one specific preset layout (even-horizontal, even-vertical, main-horizontal, main-vertical, tiled).
On my machine CTRL+B ESC 15 seems to do the trick instead.

Move panes around

CTRL+B CTRL+o to rotate them.
CTRL+B CTRL+{ and CTRL+B CTRL+} to move the active pane left/up or right/down.

Advertisements

Using Magic Cookies to run programs remotely as root

airfryer-10-minute-smartie-cookies

Some magic cookies

Unbelievable how many times I fell for this – and am still falling.

The situation is as follows: you are on a remote box, using SSH and X-forwarding enabled. You can run any graphical program (say, wireshark) as that user, but as soon as you try prepending sudo you get: (wireshark:8881): Gtk-WARNING **: cannot open display: .

If you’ve been following me for long enough, you know I’ve been bitten already by a similar problem in the past. The only (minor) difference is that this time I don’t even have a DISPLAY variable set (as root).

So here’s another fix, this time using magic cookies.

Step 1, as normal user type echo $(xauth list ${DISPLAY#localhost}). You’ll get something like this back: machine/unix:25 MIT-MAGIC-COOKIE-1 41f6c7f04a706ca5e490b3edf8a26491

Step 2, as root, run xauth add followed by the line you got as output on the shell, that is: xauth add machine/unix:25 MIT-MAGIC-COOKIE-1 41f6c7f04a706ca5e490b3edf8a26491.

Exit the root shell, confidently type sudo DISPLAY="localhost:25.0"
wireshark
and enjoy!

Removing latex commands using Python “re” module

Recently I had to sanitize lines in a .tex file where a \textcolor command had been used.
The command was being used the following way: {\textcolor{some_color}{text to color}}.

The main problem was that the command could have appeared any number of times in a line, so I couldn’t apply the command a set number of times.
Also, given any color could have been used, a simple “blind replace” was clearly not a good weapon in this case.

I therefore resorted to applying a reg ex recursively until the line was cleaned of any \textcolor command.

In a nutshell:

def discolor(line):
    regex = re.compile('(.*?){\textcolor\{.*?\}(\{.*?\})\}(.*)')
    while True:
        try:
            line = ''.join(re.search(regex, line).groups())
        except AttributeError:
            return line

The key part here is that we match not only the text inside the \textcolor command, but also what comes before and after (the two (.*?) blocks). We return them all until there are any left: when that happens, accessing .groups() will raise an AttributeError, which we catch and use as sentinel to know when to return.

Changes to RunMe.sh to root a Kindle Fire from Linux

For once, a post that’s not about testing: it’ll contain some brief notes that come in handy if you’re trying to use Root_with_Restore_by_Bin4ry_v33 on Linux to root a Kindle Fire.
The script, RunMe.sh, won’t run out of the box. To make it work you’ll need to:

  • Make the script, and the files under stuff/ executable: chmod -r 755 RunMe.sh stuff
  • Edit RunMe.sh and
    • add a shebang on the first line: #!/bin/bash. This will fix the error that says read: Illegal option -n
    • Replace wait with sleep. This will fix the error that goes wait: pid 10 is not a child of this shell
  • Replace the adb binary in stuff/ with a more recent one. Run ./adb version to check the version. Android Debug Bridge version 1.0.39 – Revision 3db08f2c6889-android should work fine. This fixes the problem with mounting/remounting the filesystem of the device. The error message misleadingly suggests that you might not have root permissions (specifically, mount : permission denied (are you root ?)).
  • Edit (or create) ~/.android/adb_usb.ini and add the USB ID for the vendor, Amazon. Just add the number in the following format 0xnnnn (e.g. 0x1949). The ID can be found running dmesg after connecting the Kindle to the machine. This fixes the problem whereby the adb server does not see any device connected (even though the OS sees it).

If all goes well, RunMe.sh should just work at this point.

Of course, YMMV and in any case I’ve got no liability if you screw it up. Happy rooting!

sort -k

TIL, when you call sort -k3, you’re not just sorting by the third field, but by whatever the value between the third field up to the end of the line is.
Not only that, in the case of ties, by default it will use also the first field.

Consider this example.

$ cat data
theta AAA 2
gamma AAA 2
alpha BBB 2
alpha AAA 3

Sorting with -k2 gives:

$ sort data -k2 --debug
sort: using simple byte comparison
gamma AAA 2
     ______
___________
theta AAA 2
     ______
___________
alpha AAA 3
     ______
___________
alpha BBB 2 
     _______
____________

Notice I’ve also added --debug, to show which parts are used in the comparisons.
So, first comes “AAA 2”, then “AAA 3”.
Also, for the two lines that have “AAA 2”, the first field is used, so “gamma” comes before “theta”.

Forget about the ties for now.
To consider field 2 only, rather than field 2 and all following fields, you need to specify a stop. This is done by adding “,2” to the -k switch. More in general, -km,n means “sort by field m up to n, boundaries included”.

$ sort data -k2,2 --debug
sort: using simple byte comparison
alpha AAA 3
     ____
___________
gamma AAA 2
     ____
___________
theta AAA 2
     ____
___________
alpha BBB 2 
     ____
____________

As you can see, field 2 only is taken into account at first.
“AAA 3” comes before “AAA 2” because, being a tie, the first field is used as a second comparison.

Taking this a step further, to actually only consider field 2 and resort to the original order in case of ties, that is, to have a stable sort, you need to pass the -s switch.

$ sort data -k2,2 -s --debug
sort: using simple byte comparison
theta AAA 2
     ____
gamma AAA 2
     ____
alpha AAA 3
     ____
alpha BBB 2 
     ____

This look similar to the first snippet, but actually the first two lines in the output are swapped. Here they appear in the original order.

Remote tail

You might have wondered why I felt the urge to specify “local” in the title of last post. Well, fast forward a few days since then, for a similar set of tests I also needed to check a log file on a remote Linux machine – that is, I needed some kind of remote tail.

ssh tail -f

We already know select will be part of our tool set. On top of that, we’ll need to forward the command across the network – and a nice way of doing that is over SSH. In Python, this task is relatively simple if you choose to use paramiko, a third party library implementing the SSHv2 protocol.

A few caveats here, as well. The following snippet is a raw prototype to demonstrate the functionality. It fit my bills, but YMMV. Of course many aspects can be improved, starting from instance with isBeginningOfMessage, which is much better placed in a derived class, so that different BOM patterns can be handled. Closing the SSH channel cleanly is also something you might want to polish before using this class.

import paramiko
import re
import select
import Queue


class SSHTail(object):
    """Tail a remote file and store new messages in a queue
    """
    READ_ONLY = select.POLLIN | select.POLLPRI | select.POLLHUP | select.POLLERR
    TIMEOUT = 1000  # milliseconds
    BUF_SIZE = 1024
    NEWLINE_CHARS = {'\n', '\r'}

    def __init__(self, host, path):
        self.host = host
        self.path = path
        self.poller = select.poll()
        self.messageQueue = Queue.deque()

    def start(self):
        """Start the tail command and return the queue used to
        store read messages
        """
        client = paramiko.SSHClient()
        client.load_system_host_keys()
        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        client.connect(self.host)
        self.client = client

        transport = self.client.get_transport()
        transport.set_keepalive(1)
        self.transport = transport

        channel = self.transport.open_session()
        channel.exec_command("tail -F %s" % self.path)
        self.channel = channel

        self.poller.register(self.channel, self.READ_ONLY)

        return self.messageQueue

    @staticmethod
    def isBeginningOfMessage(line):
        """Return True if the line starts with the hardcoded Beginning of
        Message pattern
        """
        BOMPattern = ''
        return re.match(BOMPattern, line)

    def loop(self):
        """Whilst the SSH tunnel is active, keep polling for new
        content and call parseBuffer() to parse it into messages
        """
        while self.transport.is_active():
            events = self.poller.poll(self.TIMEOUT)
            for fd, flag in events:
                if flag & (select.POLLIN | select.POLLPRI):
                    buf = self.channel.recv(self.BUF_SIZE)
                    self.parseBuffer(buf)

    def parseBuffer(self, buf):
        """Given a buffer buf, split it into messages and glue it together to
        previous messages, if buf is not the beginning of a message.
        
        Note: assumes each message is on its on line.
        """
        if buf:
            messages = buf.splitlines()

            oldest = messages[0]
            if not self.isBeginningOfMessage(oldest):
                try:
                    messages[0] = self.messageQueue.popleft() + oldest
                except IndexError:
                    pass

            for message in messages:
                self.messageQueue.appendleft(message)

Local tail

tail -f

A while ago I needed, for one of my tests, to monitor a log file on a Linux system and store any new lines, so that I could access the added content at the end of the test. In a sense, I needed a kind of buffered tail -f on a local file.

A quick search led me to the select module.
Without further ado, here’s the code to watch one or more files, and to store anything added to those files in a message queue.

It’s a quick and dirty version which can be improved in many ways. For starters, the keys to access the message queues are the sockets themselves, pretty useless in general, but good enough in my case. Second, notice the file is never closed explicitly: definitely not ideal.

import Queue
import select


class Watcher(object):
    TIMEOUT = 1000
    READ_ONLY = select.POLLIN | select.POLLPRI | select.POLLHUP | select.POLLERR

    def __init__(self):
        """Initialize the Watcher"""
        self.poller = select.poll()

        self.fd_to_socket = {}
        self.message_queues = {}

    def addFile(self, path):
        """Add a file to monitor.
        :path: absolute path of the file, including the filename
        """
        f = open(path)
        self.poller.register(f, self.READ_ONLY)
        self.fd_to_socket[f.fileno()] = f 
        self.message_queues[f] = Queue.deque()

    def start(self):
        """Start polling files"""
        while True:
            events = self.poller.poll(self.TIMEOUT)
            for fd, flag in events:
                s = self.fd_to_socket[fd]
                if flag & (select.POLLIN | select.POLLPRI):
                    lines = s.readlines()
                    if lines:
                        self.message_queues[s].appendleft(*lines)

Filtered tail in less

Not too cryptic title for a simple post on how to tail a file in less, while also filtering only the lines you are interested in.

First, less +F the file. Next, CTRL+C to stop tailing and press & to call the filtering command. Type in your pattern, hit return and resume tailing by pressing Shift+F.

Misconfigured Python pretty printers in GDB

I noticed that running an executable in gdb displayed an error – and only the first time you run the program. The error reads:

(gdb) r
Starting program: /home/chris/workinprogress/cpp/book/a.out 
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in 
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
[Inferior 1 (process 7215) exited normally]
(gdb)

The problem is, the libstdcxx directory is not in the path.
The very simple fix is to add the directory to the Python path in gdbinit.

$ cat ~/.gdbinit
python
import sys
sys.path.insert(0, '/usr/share/gcc-4.8/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end

The journey from a crash logged in /var/log/messages to the offending line in the code

Recently I had to investigate a process crash. Usually, I grab the generated coredump and feed it to gdb. Usually, this works. Unless someone has already deleted the core file by mistake.

Fear not, there’s a solution. And here I am to document it before I forget it and spend another half an hour on Google.
Look in /var/log/messages the line that references the crash:

> sudo cat /var/log/messages | grep binary_that_crashed
Feb 30 17:20:22 zeus kernel: [9232272.493642] binary_that_crashed[12868] trap divide error ip:7f11de8d4095 sp:7f11daf7d990 error:0 in libsomesharedlib.so[7f11de876000+88000]

Then work out the location in the code by subtracting the address where the lib was loaded from the address of the instruction pointer:

> python -c "print '0x%x' % (0x7f11de8d4095-0x7f11de876000)"
0x5e095

Finally, get the offending line with either addr2line:

> addr2line 0x5e095 -fe /usr/lib/libsomesharedlib.so
_ZN3XYZ14ThisMethodName4plusEv
/builds/rev1232/source/cpp/libsomesharedlib/filename.cpp:539

where the mangled name of the method, _ZN3XYZ14ThisMethodName4plusEv can be demangled using c++filt:

> c++filt _ZN3XYZ14ThisMethodName4plusEv
XYZ::ThisMethodName::plus()

or objdump:

> objdump -DCgl libsomesharedlib.so | grep -B1 5e095
/builds/rev1232/source/cpp/libsomesharedlib/filename.cpp:141
   5e095:  48 f7 f1             div    %rcx

And while we’re more or less on topic, here’s a not-too-random selection of pointers to the stack (ha!):