@Kiwibrew's Tools & Tips

Let's Build a Little UFB ISP

Sun, 31 Jul 2011 23:50:58 -0400

For an NZTelco post on the economics of Ultrafast Broadband CIRs, I put together a straw-man ISP. I pulled together what I thought was the minimum an ISP could spend to satisfy a thousand 50mbps full-duplex UFB customers with a 2.5mbps CIR.

In this ISP, we don’t have any real provisioning, OSS, or active traffic management systems. Those things are expensive, and we’re a low-cost operation. It’s all Ethernet, so we’ll have a single /22 of address space, a single gateway (also functioning as our BGP router), and we’ll add vlans and queues to the switches by hand, keeping track of everything in a wiki.

The lowest data cap will be 150GB/month, so we’ll run a Nagios process on our NMS server to alert the helpdesk on any particular queue counting more than 150GB of aggregate traffic. When that happens, we’ll ring up the subscriber to have a whinge about changing plans (to 250GB) or buggering off to Xtra. This saves a lot of time and money, as long as we assume our customers are little old ladies who don’t know anything about bittorrent or Usenet.

I’ve stuck a Packeteer capable of 620mbps in between the BGP router and the international transit. It’s too expensive to buy one that will be nice to all traffic, and buying more national bandwidth should be cheaper than buying a higher capacity shaper. International traffic is still punitively priced, bittorrent does exist, and we don’t want a few punters killing the experience for the rest, so the shaper is still a must have.

The router only supports Gig ports, so we’ll bond three 1gbps connections between the switch at Sky Tower and the router, which will have a total of 8 1gbps ports.

I’ve put in OOB management with a Telecom stick in case we do something stupid like drop vlan1 and create a spanning tree loop, thus taking down half the Internet with 700,000 packets per second of rubbish. It could happen, you know.

I’m assuming a three-man band, using a UFB tail for access and admin, again falling back to a T-Stick if it all goes horribly wrong. These three guys each get a cheap and nasty laptop, and they use Wifi in the office already built in to their Access Router. No price for that, it’s included free with the install.

All the gear is cold spared, including the UPS (they do blow up occasionally) so we’re not completely screwed if something breaks. Only if it breaks at 3AM on a weekend when we’re all too drunk to drive to the POP with the spares.

Bits and pieces are assumed to have fallen off the back of a truck, and we’ll assume someone at NHR has accidentally shipped some 10gbps optics with the Ciscos. Everything else has come from Ebay. Here’s the damage:

Cisco 3750E-24TD (used) three units: $16,740
Juniper M7i Router (repossessed) two units: $16,834
OpenGear CM4008 (new) three units: $3108
HP DL380 Rack Servers (ex-lease) three units: $7350
Packeteer PS1000-L620M (used): two units: $22,520
Compaq CQ57-104TU Notebook (new) four units: $2796
Libert 10kVA UPS (new) four units: $27,000

Total price for this ISP adventure: $96,348

If you have a little equity in your house, UDC may help you out with a finance lease. Their number is 0800 500 832, tell them Jon sent you & I may get some rugby tickets out of it. Have fun!

P.S. No whining about the UPS cost, unless you want to walk 48 floors up Sky Tower the next time the power goes out.

Calculating An Accidental Passive Reflector

Wed, 15 Jun 2011 23:20:22 -0400

Every once in a while, radio folk come up with strange, seemingly inexplicable problems. This week I was asked by a colleague how one of his ultra low-power general user licensed radios could be interfering with a 5.6GHz licensed service around 25 kilometers away. A quick look at the problem yielded a real mystery.

There’s a mountain between his antennas and the victim service
His link transmits at an angle to the receiving antenna
The power is so low (only one watt after antenna gain) that even with line of sight, there shouldn’t be a problem - due to the angle offset

Let’s do the calculations to see what sort of havoc he could be causing by pointing his transmitter at the victim with direct line of sight over a 25km path.

PTx = Power of the Transmitter: 18dBm
LTx = Loss at the Transmit End: -2dB
GTx = Transmit Antenna Gain: 16dBi
LFS = Free Space Loss 20*log[4∏*distance/wavelength] : -135dB
LM = Miscellaneous Loss: -2dB
GRx = Receive Antenna Gain: 45dBi
LRx = Loss at the Receive End: -2dB
PRx = Power received at the victim: -60dB

That’s a pretty decent signal, one that would definitely cause an issue - so if the antennas were in line, and if there wasn’t a mountain in the way, he could be making a mess of things.

Still assuming line of sight (i.e. no mountain), let’s correct the antenna gains for the angle offset and see what we get. Since antennas concentrate transmit power and receive gain in a directional beam, taking them out of alignment rapidly decreases their gain.

PTx = Power of the Transmitter: 18dBm
LTx = Loss at the Transmit End: -2dB
GTx = Transmit Antenna Gain: 6dBi
LFS = Free Space Loss 20*log[4∏*distance/wavelength] : -135dB
LM = Miscellaneous Loss: -2dB
GRx = Receive Antenna Gain: 9dBi
LRx = Loss at the Receive End: -2dB
PRx = Power received at the victim: -108

At this level, even with no mountain in the way, there wouldn’t be any interference to the licensed service. -108 is so low it’s considered background noise.

With all that taken in to account, he still worked with the MED on resolving the issue. A change in transmit frequency from him resolved the problem immediately. So what was happening?

My best guess is that his link had an accidental passive reflector, by way of an office building about 5km from his transmitter facing away from the victim service. The diagram below illustrates how this can work.

Passive reflector systems take some effort to design, so accidents like this are pretty uncommon. Angle tolerances are tight, and the reflector surface has to be really big to overcome the two independent free space radio paths that occur when they’re used. We’ve now got a first path of 5km and a second path of 30km. The passive reflector system calculation - if the angles for the main lobes of both transmitters are at just the right angle - looks like below.

PTx = Power of the Transmitter: 18dBm
LTx = Loss at the Transmit End: -2dB
GTx = Transmit Antenna Gain: 16dBi
LFS1 = Free Space Loss 20*log[4∏*distance/wavelength] : -121dB
GP = Gain of Passive Gp 20*log [(4pA cosα)/l2]: 107dBi (8x10M surface)
LFS2 = Free Space Loss 20*log[4∏*distance/wavelength] : -137dB
LM = Miscellaneous Loss: -2dB
GRx = Receive Antenna Gain: 45dBi
LRx = Loss at the Receive End: -2dB
PRx = Power received at the victim: -78

It works! Even with a double path loss, a reflector as small as 8x10M in the path could be causing the interference.

With the frequency changed, the problem was solved, but still a mystery. A passive reflector is assumed to be the cause of the mystery, but it would take a couple of days with a spectrum analyzer to prove it out. In the meantime, I’ve had a new look in to passive reflectors and have some neat ideas to play with.

Wellington Panorama, 3 May 2011 09:00 NZST

Tue, 03 May 2011 06:11:00 -0400

I woke up this morning to a stunning view of Wellington and quickly grabbed the camera. I couldn’t be bothered getting dressed and heading to the roof, and my brain was too fuzzy to remember the tripod in the closet, so I stuck the camera out the living room and bedroom windows to snap a few shots.

My first go at a panorama was using Photoshop Elements 9 for the Mac. I let Photomerge fly and came up with a huge jpg. Unfortunately I’d shot in RAW, and Photomerge had ignored all the metadata in the accompanying XMP files, so the first pass didn’t come out too hot. I compounded the problem by uploading to twitpic and the result was less than stunning. The preview was too small, and the “full size” image was too large to be useful. (15025 × 2647)

This evening I had another crack at it. I converted all the RAW files to Photoshop psd files, thus embedding the metadata from the XMPs. I then threw Photomerge at the stitching task again and was a lot happer with the result. I still had a huge photo, and so set about trying to find out how to make a zoomable panorama. Google delivered me to the perfect blog post, aptly entitled How to create a panorama and display it using the Google maps API using free software. Yep, that’s what I was looking for.

Since I’d already taken the photos (oops on the blogger’s great photography suggestions) and stitched them together, all I had left to do was create a Google Maps Panorama. The entailed using a Java program called “GMapImageCutter”, published by a nice man at the UCL Centre for Advanced Spatial Analysis. He helpfully suggested replacing his contact details in the completed HTML file before uploading to the web, so I no longer know his name. Once I set the shell script that starts the java app to chmod +x, the application popped up and was ready to go. Accepting default settings, I was within a minute or two presented with an HTML file and an image directory with 5,461 small jpg tiles.

Once uploaded to a friendly web server (not tumblr), I had to generate a Google Maps API key. This is because the generated panorama uses the Google Maps API (v2 in this case) for all the neat zooming and panning over the tiles. I then inserted an iframe into my Tumblr post (directly below) and it was all go.

This is probably not the end of the line for me with panoramas. I like them, and I’d like to do them better, so watch this space.

ESRI Shapefile (Points) to Google Fusion Tables

Thu, 28 Apr 2011 00:03:00 -0400

Task of the day is to open an ESRI shapefile containing point data, clean the data, and upload it to a Google Fusion Table. I’ll be doing this in MacOS, but the tools in use are also available on Windows and Linux.

First step is to convert the data to a format that can be read by Google Refine. Open the ESRI .dbf file (from your shapefile folder) in OpenOffice 3, and then save as CSV.

Open with Google Refine to start processing. A quick glance at the data shows that fifteen rows have no location data. A numeric facet on the X coordinate allows me to find the offending rows, and “remove all matching rows” gets rid of them.

Another look at the data shows that all the location data is in NZMG - the New Zealand Map Grid format. This format isn’t useful to applications like Google Fusion Tables, so we’ll have to convert it via LINZ conversion tools.

The LINZ conversion tool requires location data to be a coordinate set per line separated by a comma, and the existing data has the coordinates in columns, so we’ll make a new single column with both x & y coordinates in it.

In Refine, we choose “add a column based on this column” from the menu above our x coordinates. Then concatenate the columns with this command: “cells[“y-coords”].value + ” ” + cells[“x-coords”].value”

For simplicity of converting the data using the LINZ web page, I exported a new CSV from refine, opened in OpenOffice, then copied the column to convert. After pasting the converted coordinates back into the CSV file, it’s ready to upload into Fusion Tables - as a CSV. Once in, it looks like this:

A couple of quick tweaks to columns are required in Fusion Tables. Firs set place name column type to “text” to prevent Google trying to geo-code city names, then set location column type to “location” so that Google places marks via lat/long.

With these two steps done, the data is ready to visualize!