KickLogstash.sh: 7 Lines to Restart Logstash

It might well be my naive, beginner’s implementation of it, but it sure seems like Logstash is pretty easy to confuse. Elasticsearch temporarily unavailable? Hang. Some weird formatting sneaks in? Hang. Some as-yet-undiscovered circumstance that causes java to eat all your memory? Hang. And each time it hangs, Logstash becomes unresponsive, never recovering. So for the moment, I’ve given up, and I’m resorting to a dumb, sledgehammer-like solution:

#! /bin/sh
FILELENGTH=`stat --format=%s /var/log/logstash/logstash.err`

if [ $FILELENGTH != "0" ]
then
        killall -9 -u logstash
        service logstash restart
        logger Logstash Killed and Restarted Due To Non-Zero Error Log
fi

Again, I’ve only had about a week of actually sending a lot of input to Logstash, but so far, there’s always something in the error log when it hangs, and there’s rarely anything in there when things are going well. So this barely-a-script checks the size of the error log, and if it’s non-zero, it SIGKILLs every process belonging to the logstash user, and then tries to start Logstash. The restart part assumes you have a Logstash script in init.d, which I certainly hope will be included as part of a proper .deb distribution of Logstash someday soon. I run it every five minutes with cron:

*/5 * * * * /etc/logstash/kick_logstash.sh

Coping With EXIF Rotation Problems

At Knowledge Architecture, we may not operate at the scale of Facebook or Flickr, but Synthesis sees enough image uploads that we’ve run into some pretty strange image rotation problems now and then. I’ve spend a lot of time with the issue, and while I’m not naive enough to declare victory over something as devilish as EXIF rotation gremlins, we’ve got an update rolling out this week that will at least avoid the species we’ve seen so far.

First, a little background, and we’ll imagine a portrait-orientation photo of the Sutro Tower as our example image. Digital images are stored in a given rotation, which generally corresponds to the default rotation of the image sensor—so our Sutro Tower photo is probably sideways on disk. If we open the image in any half-decent viewer, though, the tower will be right-side up because most digital cameras annotate image files using the EXIF Orientation tag. That’s used by most image-viewing things to recognize situations where image may be stored on disk like this, but when displayed it should be (say) flipped, and rotated 90° to look like that.

It’s gotten better, but support for that Orientation value isn’t perfect. So when someone uploads an image to Synthesis, we look for the Orientation EXIF field, rotate the image data itself so that it’s stored the way it should be displayed, and then we remove the Orientation field, because its instruction to rotate the image on display is no longer needed.

The problem we’ve run into is that some image editors apparently don’t update EXIF data properly; they rotate the image data itself (so the Sutro Tower is right-side up), but leave the Orientation flag in place. So our uploading process would dutifullly rotate the image one more time, and the tower would end up pointing sideways. It was a frustrating situation.

Luckily, those EXIF-unaware image editors seem to leave all the other EXIF data unchanged, too. So what our solution does is compare the so-called “EXIF Width” and “EXIF Height” field values to the width and height of the image data on disk. If it looks like the image has already been rotated, we simply remove the Orientation field.

So here’s the important part of the code. In short, we:

  • calculate the ratio of actual image width to height, and the ratio of EXIF-reported height to width,
  • round them both to one decimal place (to account for resized images subtly changing that ratio),
  • and check to see if one equals the other. If so, the image has been rotated already.

This is C# but it’s mostly math so it should be easy enough to port to whatever language you’re using. Not shown is fetching the orientation, exifWidth, and exifHeight values; they’re EXIF fields 0x0112, 0xa002 and 0xa003, respectively. I found this list of EXIF tags very helpful during this whole process.

// Useless data check (this check doesn't work on squares)
if (exifWidth > 0 && exifHeight > 0 && exifWidth != exifHeight)
{
    // To see if the bits were rotated, we calculate a height/width ratio, because 
    // we might be dealing with a resized image. We round, becuase resizing might 
    // slightly change this ratio
	
    decimal actualRatio = Math.Round(
		((decimal)img.Height / (decimal)img.Width)
		, 1); // height / width
		
    decimal exifRatio = Math.Round(
		((decimal)exifWidth / (decimal)exifHeight)
		, 1);  // width / height

    if ( orientation >= 5 && actualRatio == exifRatio )
    {
        // SO, if the orientation is rotate90 or rotate270 (with or without flip),
        // and the image seems to have already been rotated 90 or 270 (the width 
	// is now the height), we're going to assume that some bad software rotated 
	// the pixels and never removed the orientation flag. In such a case, we'll
	// trust the lousy software's rotation, and remove the flag.

        orientationPropItem.Value = new byte[1] { 1 };
        img.SetPropertyItem(orientationPropItem);

        // We return true because in our implementation, we just need to know if the 
	// image file we're storing has changed at all, whether that means rotation 
	// or just removing an EXIF value.
        return true;
    }
}

What I Like Is Looking At My Phone

The other day I was thinking about the dawning of The Age of Smart Watches, and trying to answer a skeptical friend’s “why?” Why do people think they want such a thing?  Or, at least, why do companies think that people want such a thing? I ran through the usual suspects: the wrist is a convenient place to put always-on health sensors, it’s a place where (unlike your face) it’s already socially acceptable to wear some kind of gadget, it can give you feedback (or directions) without requiring you to wear headphones or look at a screen, etc.

I thought my big finish would win her over: smart watches will help people to stop being Phone Jerks. People know it’s obnoxious to keeping pulling out their phone to check on some alert, you see, and a smart watch will let them deal with those interruptions quickly and unobtrusively! Years from now we’ll look back on this whole embarrassing Age of Smart Phones as just an impolite blip in history.

She didn’t buy it, though, and rightly so. And the reason why is the same reason why I’m so happy that my wife and I never bought a video baby monitor.

A Brief Digression On Video Baby Monitors

If you haven’t looked into the list of accessories considered near-mandatory for baby-raising lately, you might be surprised at how common a home infant sleep surveillance setup is. The only downsides usually mentioned are cost, complexity/fiddliness, and the potential for some hacker to spy on your little one. The advantage boils down to: you can quickly and silently verify that your baby is still breathing! That’s no joke, and in addition to solving those paranoid “…a little too quiet…” moments, this benefit is even an important part of some sleep training schemes.

Fortunately, we found some advice online that described the greatest downside of a video monitor: that you will never be able to look away. What’s billed as a way to quickly soothe those sudden, irrational worries instead becomes its own necessity; as soon as the way you know your baby is safe is by watching her on a video screen, you must always watch your baby on a video screen. Of course, new parents love to just look at their infants. Once you have one (a video monitor, that is), I’m sure that most of the time it just feels nice, and comforting, to watch a live stream of your infant sleeping.

Remember BlackBerry?

If that example feels more tangential than parallel, consider that eons ago, Blackberry devices, Palm Pilots, and other proto-smartphones were the heroes we were counting on to liberate us from our desktop email clients—go ahead, leave at 5:00 tonight, be with your family, and if something important comes up, you’ll know! It didn’t take long for experience to complicate such promises; a cursory Google turns up warnings as early as 2001 that, “your work hours suddenly are extended from finite to infinite.

Just as we were right to be skeptical of the benefits of parental voyeurism and email gadgets, we should be skeptical of claims that smart watches will do anything but increase the amount of time we spend staring at screens. Because the truth is, we like playing with our gadgets. I don’t have notifications turned on for emails, and I generally keep clear of Facebook, but I have my Twitter feed crafted to ensure that whenever I need to look at my phone for a moment, there’s something waiting for me. Sure, I enjoy the thinkfluencers and whatever else, but if Twitter closed tomorrow I’d just find something else, because what I like is looking at my phone.

Traveling With a Baby in Italy

Several months ago, my wife and I completed a two-week trip to Italy with our then seven month-old son, Felix, and for the most part, it was great! Before we left, though, we looked around for guidance and didn’t find much that lined up with our idea of a vacation: a “last” trip before Felix could walk (let alone express an opinion about our itinerary), where we’d take a few day trips, walk a lot, and experience life in a new city or two in addition to checking off a few postcard sites on our lists.

So I wanted to post about some of what worked, and what didn’t work, for us. My TL;DR advice is that you should absolutely travel with your infant, too—I treasure my memories of this trip more each day. That said, every baby (and every family) is different, so I hope my notes will help you to draw your own conclusions.

A quick itinerary outline:

  • We flew in to Florence, taking the tourist bus in to the city.  We stayed in an AirBnB for 10 days.
  • We went to Assisi for one night—and we actually would’ve liked more time there.  We stayed outside of the city, which was kind of a pain: we needed a ride from the train station, which my Italian (and our host’s English) was only barely good enough to arrange, and of course there was no infant car seat, which I guess is legal (?) but obviously isn’t ideal. Stay in the old city itself, and you can take a tourist bus to and from the train station.
  • We stayed in Lucca for one night, spur of the moment, mostly because the focaccia place I wanted to go to was closed on the day we arrived. No, I’m not kidding, and yes, Forno a Vapore Amedeo Giusti was worth it.
  • We took a day trip to Venice: the earliest train up in the morning, and the last train back at night.
  • Those trips were all during our ten days in Florence.  So we double-paid for the Assisi and Lucca overnights, but staying that long got us a discount, and it was great to be able to leave most of our stuff at “home” for those short trips.
  • Then we took the train down to Rome and stayed in Testaccio (a short walk from the Roma Ostiense train station) for three nights before flying home from Fiumicino.  Testaccio is not a short walk from the Pantheon or St. Peters, but there are buses, and the walk to the Forum / Palatine Hill / Colosseum area isn’t so bad.  Testaccio is very close to the Pyrimide metro stop, so you have convenient access to commuter trains to the airport and to Ostia Antica (which was another highlight of the trip).

 

With that out of the way, on to the details:

  • As far as we saw, yes, Italians love babies. Ergo / Baby Bjorn -style carriers are relatively rare there, so a baby in one of those is extra adorable (and a little amusing), especially when a man is wearing it.
  • Babies are a great low-pressure conversation starter, and I had lots of fun using my pidgin Italian to answer questions about how old Felix was, where we were from, etc, etc.
  • The flights didn’t go all that well for us, because he didn’t sleep that much.  But we’re still glad that, when we checked in for the intercontinental flights, we asked for a bulkhead row with a little crib.  This is free, and if nothing else it gives you some extra room and place to put your baby even if she’s not sleeping, so do it!
  • We didn’t bring a stroller, and we didn’t regret it.  Spaces are tight, roads are bumpy, and you can split carrying duties.  Our baby was already used to falling asleep in our Ergo.
  • Part of what made 7 months an ideal age is that our son was still breastfeeding, almost exclusively.  So other than the occasional banana, we didn’t worry about finding appropriate food for him. Some reading I found online made me a little worried about how Italians would react to public nursing, but everyone was friendly and accommodating about it. My wife fed Felix in church squares, art galleries, ancient ruins, restaurants, trains, alleyways, etc, etc, etc.
  • Speaking of not being overly concerned with modesty, we didn’t find a whole lot changing tables in bathrooms, so that list also describes places where we changed Felix. The worst was in a traincar entryway (between compartments) on the way home from Venice, which we later learned was unnecessary: the bathrooms in the first class car at the head of the train has a changing table, and yes, 2nd class babies can still use it.
  • Although “everyone was cool with the baby” is the general theme, we were actually turned away from a restaurant. I can’t say I blame them, but after that we really scaled back our food ambitions. We tried to eat our special meals at lunch, when restaurants were a little emptier and a little more understanding.  Felix was just starting to get hard to handle at meals, though, so even that didn’t help much. In retrospect, we might have been better off just feeding him some real food to keep him happy and occupied—we didn’t do that because we were taking the injunctions to introduce foods slowly and selectively pretty seriously.  Two of our lunches we basically ate in shifts, alternating “hang outside with the baby” duty with “eating lunch”.  I still can’t believe that the wonderful people at Pallotta, in Assisi, not only put up with us, but were also incredibly kind about it.  Their food was one of the highlights of our trip.
  • Given that, very few of our dinners were in restaurants.  Instead, we got food to-go; cobbled together lunch-style meals of bread, cheese, and produce; and cooked at home.  We were definitely glad that we rented a place with a kitchen.
  • We did not rent a place with a crib, and that was OK; instead, we made him a little nest out of pillows and blankets. He didn’t sleep all that well, but then again that wasn’t unusual for him. In Assisi we had to all share a bed, which wasn’t a restful night; in Lucca he slept on the second of two beds, walled in by pillows; and in Rome (in Testaccio) they had a pack and play -style crib.
  • Our place in Florence was nice, but didn’t really have a great hanging-out area. In some ways, maybe that was a beneficial extra nudge to get out of the house. On the other hand, most of our nights we spent “in”, so in hindsight maybe we would’ve spent a little more for a balcony with a beautiful view…
  • We cloth diaper at home, but bought disposables (pannolini—”little cloths”) while we were there.  Our AirBnB had a (slow) washing machine, but come on, it’s a vacation!
  • We packed very light and didn’t regret it. For Felix, we had one warm and one cool sleep sack, a quilt, and a few toys, in addition to a few outfits.  We brought a breast pump and a bottle but I don’t think we ever used them.
  • Most of our trip was walking around, which was tiring, but left us free to stop for food / diapers / nursing / playtime at our leisure.  Honestly, our museum visits were more exhausting than the wandering.
  • It was great, but it was tiring, and we often slept poorly, so we had to preserve our sanity with the occasional solo expedition.  Often I’d leave early in the morning to walk around with Felix while my wife caught up on the sleep he’d denied her the night before.

I’ll leave it at that, but you’re welcome to comment if you have any questions I might be able to answer.  It was a challenging trip, but also an incredibly rewarding one, and I can already tell that I won’t remember the hard parts.  If you’re on the fence about traveling with a ~7 month-old, stop reading and go book some tickets!

NSRangeMake

Maybe this is common and maybe it’s not, but in the course of my work writing Objective-C for iOS, I end up using CGRectMake()—to make, yes, a CGRect—pretty frequently, but I only rarely need to build an NSRange.  So following the CGRectMake pattern, I inevitably start typing “NSRangeMake” and along the way the only method completion I get is for NSRangeFromString().  A few befuddled minutes later, I realize that what I’m looking for is NSMakeRange(), and then I can go back to slowly forgetting about that reversed word order until I run in to the same problem a month or two later.  Since I really my start programming in PHP, this kind of thing shouldn’t throw me, but it does.

So on the off chance that this happens to you, too, here you go:

#define NSRangeMake(location, length) NSMakeRange(location, length)

I wouldn’t recommend actually using NSRangeMake() in code anywhere, but this will at least give you a little reminder.

SQL Azure, Management Studio 2008, and the ‘sys.configurations’ error

Here you are, just trying to get some work done by connecting to a SQL Azure instance using Microsoft SQL Server Management Studio (hereafter “SSMS”) 2008. But instead of a connection, all you get is a line of Microsoft free verse:

Invalid object name 'sys.configurations'. (Microsoft SQL Server, Error: 208)

Googling around for help returns several instances of the same solution, that has you start by clicking the “New Query” button.  With this workaround, however, you lose the helpful hierarchical DB browser in the left pane of the window.

A better solution is to install (deep breath first) Microsoft SQL Server Server 2008 R2 RTM – Management Studio Express.  MSSSR2RMSE works with SQL Azure, the only catch is that if you choose the “Upgrade” option in the installer, your SSMS won’t be… upgraded.  Instead, you want the “New Installation or Add Features” option.  From there, you just accept the legalese, make sure “Management Tools – Basic” is selected (which it should be by default), decide whether or not to allow diagnostic info to be sent, and then leave the installer to its work.  When it’s done, you’ll find that your old SSMS has been updated to 2008 R2, no need to create new shortcuts, etc.

I should note that I followed the “upgrade” path first, which did seem to be doing something, to the point of even requiring a restart.  If you follow these instructions to skip straight to the “New Installation” and things aren’t working, try running the “upgrade”.  If it looks like the upgrade procedure is a necessary first step, please let me know, and I’ll update this post.

When Your DataMember Custom Property Names Aren’t Working

Let’s say you’re writing a WCF REST service, and in proper C# fashion, your object’s properties start with a capital letter. Then your service is being consumed by Objective-C code or javascript, which call for lowercase. Since WCF uses the DataContractJsonSerializer, you can set a different name for the property to serialize to:

    public class MyObject
    {
        [DataMember(Name = "name")]
        public string ObjectName { get; set; }

But for some reason, it’s not working; you still end up with “ObjectName: ‘Yadda'”. I ran into this issue, and the problem turned out to be a missing line of boilerplate:

[DataContract(Name="MyObject")]
public class MyObject
    {
        [DataMember(Name = "name")]
        public string ObjectName { get; set; }

As least as far as I’ve seen, a WCF REST service can chug along just fine without that DataContract parameter in most respects, but if you leave it out, you forego being able to set custom property names. Hope this saved you a headache!