Image Manipulation in Ubuntu with ImageMagick Tools

Can't believe I haven't come across convert and mogrify before now. Much time could have been saved in the past.

I used Ubuntu 10.04 Lucid for this, but I doubt it matters much as long as you have a recent Linux distribution.

sudo apt-get install imagemagick

Example: resize an image to fit in a 100x100 pixel box, and place it in the centre of a 100x100 pixel box:

convert foo.png -resize '100x100' \
  -background transparent \
  -gravity center \
  -extent '100x100' \
  bar.png

Lots more good documentation here.

Ubuntu Remote Desktop

Overview

I recently had the need to get reliable remote access to the desktop of an Ubuntu 10.04 install. There are many ways to enable remote access to your X11 desktop in Ubuntu, each with different trade-offs. Here are three (all VNC-based) that I recently looked into:

• Built-in - System > Preferences > Remote Desktop. Easy, but only works if you're already logged in locally. i.e. after a system restart you can't connect remotely until you login locally again. Common work-around for that: use auto-login (yuck).
• Xvnc - Create a new X display only viewable via VNC. i.e. the remote user has its own desktop unrelated to what is seen on the monitor if a local user logs in.
• x11vnc - Serve up what is being shown on the local monitor (i.e. a "real" X display) over VNC. Similar to the built-in solution, but without the "must be logged in" restriction.

My requirements were:

• Must be able to login remotely after a system restart
• Must be able to plug in a monitor and see what the remote user sees/last saw (i.e. be able to fix things in the GUI if I break them remotely)

What I Actually Did

Used x11vnc and started right after gdm starts. Here are all the steps:

1. sudo apt-get install x11vnc
2. Create a VNC password in a stable location:

   sudo mkdir -p /etc/x11vnc
   x11vnc -storepasswd  /etc/x11vnc/passwd

3. Add the following to /etc/gdm/Init/Default, right before the exit 0 at the end of the script:

   /usr/bin/x11vnc -safer -allow 127.0.0.1,192.168.1. -o /var/log/x11vnc.log -xkb -bg -forever -rfbauth /etc/x11vnc/passwd -display :0

4. Reboot, or just restart gdm (sudo restart gdm).
5. Should now be able to connect to port 5901 with a VNC client (I used TightVNC).

If things don't work, check /var/log/x11vnc.log -- there's usually a lot of detail in there explaining exactly what is going on.

Clusterfuck Alert

Ubuntu (at least in 10.04) has a crazy issue where X won't start if a monitor is not plugged in when the system restarts. I don't know -- and don't want to know -- why, but there are work-arounds. I'll link to a few below, but the gist of it is to not use the fancy NVidia or ATI drivers, but instead just use the VESA driver. If you're going to have a monitor attached all the time, this is not an issue.

References

• Forum thread on how to "fix" the issue around rebooting without a monitor
• Xvnc man page
• x11vnc, the most configurable command line app yet written by man
• x11vnc in Ubuntu before login

Mount NTFS drives using /etc/fstab in Ubuntu 10.04 Lucid

This is mostly a note to myself, but here's the line to add to /etc/fstab to mount an existing NTFS drive read-only in Linux (Ubuntu Lucid 10.04).

/dev/sdc2 /mnt/ntfs500 ntfs nls=utf8,uid=mark,gid=mark,dmask=007,fmask=117,ro 0 0

I used the GUI System -> Administration -> Disk Utility to figure out that the drive I wanted was /dev/sdc2.

References:

• Mounting windows partitions in Ubuntu

mkdir -p "trick"

Had no idea you could do this:

mark@pc:~/tmp/x$ mkdir -p src/{main,test}/{scala,java,resources}
mark@pc:~/tmp/x$ find .
.
./src
./src/main
./src/main/java
./src/main/resources
./src/main/scala
./src/test
./src/test/java
./src/test/resources
./src/test/scala

Awesome. Discovered here.

Update: 2011-08-25

I was wondering why this wasn't mentioned on the mkdir man page: the reason is it's a general bash thing for generating strings called brace expansion.

$ echo foo{bar,baz,boo}
foobar foobaz fooboo

Bash history, caret, search & replace

I've been a (blind) user of bash's "caret replacement" for ages, but have occasionally wondered how to replace all occurences of a word, rather than the first.

$ echo "foo bar baz foo"
foo bar baz foo
$ ^foo^quux
echo "quux bar baz foo"
quux bar baz foo

The answer is to use the more robust event designator syntax: !!:gs/search/replace/

$ echo "foo bar baz foo"
foo bar baz foo
$ !!:gs/foo/quux/
echo "quux bar baz quux"
quux bar baz quux

While on the topic of esoteric bash commands, here's another good one. You may (?) know of !$ which references the last argument of the last command in history. That's just an alias for !!:$, which shows the more general form of the command. You can access any argument of any command in history.

$ echo "test" > /tmp/blah.txt 
$ cat !!:3
cat /tmp/blah.txt
test

There's lots more. In particular, you can operate on any command in history, not just the last one (!!). See the bash manual link in the references below.

References

• History Interaction in the bash manual
• "Caret search and replace" question at StackOverflow

Scala method signature rules

I came across a great post on the IDEA Scala Plugin message board the other day. Apparently there are best-practices/conventions for writing method signatures in Scala. Adhering to these rules communicates to the user more than just argument and return types: it can tell you if a method has side-effects, or is meant to be a "property".
Check it out: Rules for Scala method signatures.
Also check out IDEA 10: the best Scala IDE.

Scala XML Gotchas

Scala's built-in XML support is perhaps flawed, but still offers very convenient syntax for simple XML manipulation. Even ignoring performance concerns and concurrency issues, there are still weird gotchas that the average user may need to deal with...

CDATA magically escaped

scala> val xml = <xml><test><![CDATA[a < b]]></test></xml>
xml: scala.xml.Elem = <xml><test>a &lt; b</test></xml>  <-- WTF?

Same when loading from a String:

scala> val xml = XML.loadString("<xml><test><![CDATA[a < b]]></test></xml>")
xml: scala.xml.Elem = <xml><test>a &lt; b</test></xml>

This is not what you want. The stuff in the CDATA is meant to be left alone. Instead, it seems that the CDATA is eaten and its contents magically escaped. This causes lots of grief if the contents of the CDATA are Javascript, for example.
One workaround is to use the built-in ConstructingParser to load XML.

scala> val xml2 = ConstructingParser.fromSource(Source.fromString("<xml><test><![CDATA[a < b]]></test></xml>"), preserveWS = true).document.docElem
xml2: scala.xml.Node = <xml><test><![CDATA[a < b]]></test></xml>

Looks good.
You can also use <xml:unparsed>. Check out this Scala XML faq for more.

XML Comments eaten

When loading XML from a string, XML comments disappear. Example:

scala> val looksGood = <xml><test><!-- comment --></test></xml>
looksGood: scala.xml.Elem = <xml><test><!-- comment --></test></xml>

scala> val wtf = XML.loadString("<xml><test><!-- comment --></test></xml>")
wtf: scala.xml.Elem = <xml><test></test></xml>

Again, ConstructingParser can fix this:

scala> val correct = ConstructingParser.fromSource(Source.fromString("<xml><test><!-- comment --></test></xml>"), preserveWS = true).document.docElem
correct: scala.xml.Node = <xml><test><!-- comment --></test></xml>

There are some alternatives if you run into these issues.

• As described above, use scala.xml.parsers.ConstructingParser to load XML
• Use the Lift web framework's PCDataMarkupParser (extends Scala's built-in MarkupParser with various improvments)
• Daniel Spiewak's Anti-XML project looks promising
• Use any of the million Java XML parsers that are out there (but give up the convenient scala.xml syntax

ssh client config: hosts

Slightly embarrassed that in more than a decade of daily ssh use I've never made use of ssh client config to simplify connecting to commonly used hosts. The idea is you can just ssh foo rather than ssh -p 12345 fluffy@foo.blah-blah.on.ca .... This is especially useful if you connect to a lot of EC2 hosts frequently and don't want to remember their ugly names (or setup DNS). Best understood by example:

Contents of ~/.ssh/config:

host ec2-webserver
    hostname ec2-123-456-78-90.compute-1.amazonaws.com
    user root
    identityfile ~/my-ec2-key.pem
    compression yes
    protocol 2

host home
    hostname my.place.com
    port 51000
    user fluffy
    identityfile ~/.ssh/id_dsa
    ServerAliveInterval 15
    ServerAliveCountMax 4
    compression yes
    protocol 2

After this is setup, you can simply type ssh ec2-webserver or ssh home rather than the full ssh command. There are a million other ssh client config options you can set, too. As expected, all the ssh tools like scp honour these settings.