tech poet - Ernesto Ongaro

Flush Jaspersoft OLAP Cache from Talend/JasperETL

2011-12-15T17:57:00.000-08:00

JaspersoftETL is a Java code generation tool designed to load your operational data into a data warehouse and get it ready for reporting and analysis. Jaspersoft OLAP takes advantage of aggregate tables and an in-memory cache to speed up queries. If you're running a nightly job to load the DW, it's a good idea to clear the OLAP cache at that time, otherwise your users will not take advantage of new data.

There are several ways to clear the OLAP cache:

Login with superuser privileges and display the OLAP Settings page.
In the OLAP Settings panel, click Flush OLAP Cache.
Using utilities such as cURL and wget, send an HTTP request to this URL: http://server:8080/olap/flush.html?j_username=superuser&j_password=
Restart the application server instance.
Call the Java API.

I've created a small component that allows the clearing of cache via the URL method from a JasperETL/Talend job. Essentially it will prompt you for a username and password (and note that only superuser can clear the cache by default) and a server URL. The URL should point to something like http://server:8080/jasperserver-pro/olap/flush.html

You may find the tJasperOLAPCache component on talendforge.org

Another useful thing to do would be to use the tJasperServerExec plugin to re-populate the cache.

Using tGoogleAnalyticsInput in JasperETL/Talend

2011-12-01T17:50:00.000-08:00

This is a short article on getting Google Analytics into JaspersoftETL/Talend ETL tools. Google Analytics has a nice interface and many great ways of analyzing web traffic, it'd be hard to beat the work Google did there. The purpose of this connector would be to bring the data into your own data warehouse and join it with other intelligence you might have. A great way to display the aggregated data would be with a BI tool like Jaspersoft.

I'm writing because the help for the tGoogleAnalyticsInput is in French and incomplete.

So let's get started. You'll need JasperETL 4.1.2 or older (commercial editions here, open source here).

Installing the plugin:

Create a new Job Design and drag the tGoogleAnalyticsInput component from the Palette (under Internet):

Configure the component. First start with finding some fields you'd like to use from the Google API: http://code.google.com/apis/analytics/docs/gdata/dimsmets/dimsmets.html. Note that you need to have some metrics (can't be all dimensions, vice versa).

Here's an example schema (you'll have to put in the fields yourself):

Now configure the component. In order to get your Profile ID (note it doesn't start with UA) read the cryptic Google Documentation about it. I got mine by examining the URL while logged into GA and looking at the &id= parameter.

The "value" column comes directly from the Google API and will also have to be filled in. I've also found that I HAVE to filter on period. You can do 1980-2020 if you want to include everything.

Now you can run the the job and see your visitors:

Funny enough. If you're reading this right now then you will eventually flow through the GA database and the very job I'm displaying above ;)

Icinga Reporting with JasperReports Server - Part 2: Usage + Basic Report Development

2011-08-01T12:14:00.000-07:00

Short tutorial, shows how to schedule a report, how to change the way you are prompted for parameters and some basic editing of reports within iReport

Icinga Reporting with JasperReports Server - Part 1: Installation

2011-08-01T11:30:00.000-07:00

A quick overview of how to install and configure the reporting add-on packages for Icinga (a Nagios Monitoring Fork), written for JasperReports Server.

Backing Up JasperServer in Linux

2011-04-18T11:59:00.000-07:00

This is a sysadmin how-to for backing up JasperReports Server (formerly JasperServer) and it's database (MySQL only for now).

I wrote a fairly robust script to achieve this task. It will read the database configuration from your default_master.properties file, work with several versions of JasperServer, delete old backups, write to the syslog and export the repository contents to file.

First drop my backup script into your \buildomatic folder if you're using 4.0 and abovie, 3.7 and below put in your /scripts folder.

Once the script is in place edit the variables at the top of the script:

This is to signify that you're using JasperServer 3.7.1 and below:
IS_PRE_4=false

This is a name for the product:
PRODUCT_NAME="JRS-4.0-Pro"

This is where backups will be stored:
BACKUP_DEST="/home/ernesto/Dropbox/Backups"

This will delete old backups if set to true
DELETE_OLD=true

How many days to keep old backups
DAYS=120

Now add it to your crontab. I will do it once a week in a lazy manner so will add the script to vim /etc/cron.weekly/js-backups

#!/bin/bash
/usr/local/jaspersoft/jasperserver4/buildomatic/js-backup.sh

The script will log to /var/log/messages:

The output files will look like this, I chose to stick them in my Dropbox folder for off-site backups:

You can download the script from this jasperforge forum post or view it below:

Code snippet - JasperServer Backup Script on Snipplr

Music Player Daemon w/ lame Support in Debian

2011-04-01T00:28:00.000-07:00

Trying to get my old Blackberry to stream from my mpd server, it turns out the built in http stream output is not compiled with lame encoder (mp3) support. Here are some instructions on building the source package with the lame encoder options compiled in for Debian Squeeze (6.0)

(as root)
Install libmp3lame0 and lame from http://debian-multimedia.org/

cd /tmp/

Get the build dependencies for mpd:

apt-get build-dep mpd

Get the mpd source package:

apt-get source mpd

cd mpd-0.15.12 (your version may differ)

Edit the file debian/rules and remove --disable-lame
Build the binary:

./debian/rules binary

Build the .deb package:

cd ..

dpkg-source -b mpd-0.15.12

Install the package:

dpkg -i mpd_0.15.12-1.1_amd64.deb

Lock the package in place so that updates won't blow away your change:

echo mpd hold | dpkg --set-selections

That's it! Then you can configure an audio output in /etc/mpd.conf:

audio_output {
   type "httpd"
   name "My HTTP Stream"
   encoder "lame"
   port "8000"
   quality "5.0"
   always_on "yes" #This option doesn't seem to work for me :(
}

ThinkPad Overheating Issue Fix

2011-03-24T16:58:00.000-07:00

She's pretty hot!

On my ThinkPad T410s fans were disabled by default after installing Debian 6 (Squeeze).

I would randomly be working on CPU intensive stuff and the computer would shut down with:

Critical temperature reached (128 C)

To fix this I created the file /etc/modprobe.d/thinkpad_acpi.conf and added options thinkpad_acpi fan_control=1 to it. I've submitted a bug against the acpi package in Debian.

I got help from ThinkWiki on this.

File Management Tip for Nautilus/Gnome in Debian

2011-03-09T14:08:00.000-08:00

nautilus-open-terminal in action

Often I find myself browsing to a directory in Nautilus and then realizing that I'd rather be in terminal. Other times I want to be in Nautilus when I'm in the terminal. There's certain things the command line is better at than the file browser and vice versa. Switching between the two is fairly easy in Gnome/Nautilus.

Going from Nautilus to Terminal:
You'll need the nautilus-open-terminal package ( sudo apt-get install nautilus-open-terminal; killall -HUP nautilus ). Then in any Nautilus, right click on white space and select "Open in Terminal"

Going from Terminal to Nautilus:
This one is simpler. I watched a guy at work do this on his Mac and got jealous. You can go from Terminal to Nautilus opened in your current directory by typing: nautilus .

ernesto@debian-t410s:~$ nautilus .

Linux zd1211 Wireless Speed Fix

2011-01-16T20:15:00.000-08:00

For some reason my Belkin 54Mbps USB wifi adapter operates at 1 Mb/s under Linux.
It's detected as the following by lsusb:

Bus 001 Device 002: ID 050d:705c Belkin Components F5D7050 Wireless G Adapter v4000 [Zydas ZD1211B]

If I do a iwconfig wlan0 rate 54M the card starts working at 54Mb/s. To fix this more permanently I added the following to my /etc/network/interfaces file:

auto wlan0
iface wlan0 inet dhcp
        wpa-ssid 56k
        wpa-psk secret
        wireless-rate 54M

Solaris on Virtualbox

2010-12-07T13:26:00.000-08:00

Trying out Solaris on Virtualbox to troubleshoot an issue at work with no luck booting. The VM would hang after the grub prompt.

Disabling USB support and sound support did it for me.

Cron Jobs in the Green Data Center

2010-11-05T14:02:00.000-07:00

Yesterday I wrote a script to monitor a UPS (running apcupdsd) server using Nagios. Today I'm looking at some of the results of the performance data. It seems that the hourly cron jobs on my server actually cause some power usage. I'll have to think over how often it's actually necessary to run those scripts given it's sucking a noticable bit of juice. Check out the graph (produced by PNP for Nagios)

4 hour Load Percentage of our new APC 2200 @ work

SSH Client Keep Alives

2010-10-07T11:38:00.000-07:00

At my new office in the Shell Building the main router is a SonicWall which has a setting to drop idle TCP connections after 15 minutes. This is annoying since I usually have several going at once. I don't have access to the router so I had to work around the issue.

The poor man's solution is to run the "top" command on your idle terminals so that packets come through. Not elegant and too manual for me.

The rich man's solution is to change the server's keep alive settings (TCPKeepAlive yes). I call this the rich man's solution because it means every one bends over for you.

The middle class man's solution is to add some directives to the default SSH client options on your local machine. Here's how:

Edit ( as root) /etc/ssh/ssh_config and add the following lines:

TCPKeepAlive yes
ServerAliveInterval 60
ServerAliveCountMax 5

This means that every 60 seconds your machine will send a keep alive to the server if there is no other activity. If the physical connection gets dropped then every 60 seconds the keep alive will be sent until it's done that 5 times, finally giving up (in 5 minutes).

From the man page:

TCPKeepAlive

Specifies whether the system should send TCP keepalive messages to the other side. If they are sent, death of the connection or crash of one of the machines will be properly noticed. However, this means that connections will die if the route is down temporarily, and some people find it annoying. The default is ``yes'' (to send TCP keepalive messages), and the client will notice if the network goes down or the remote host dies. This is important in scripts, and many users want it too. To disable TCP keepalive messages, the value should be set to ``no''.
ServerAliveCountMax

Sets the number of server alive messages (see below) which may be sent without ssh(1) receiving any messages back from the server. If this threshold is reached while server alive messages are being sent, ssh will disconnect from the server, terminating the session. It is important to note that the use of server alive messages is very different from TCPKeepAlive (below). The server alive messages are sent through the encrypted channel and therefore will not be spoofable. The TCP keepalive option enabled by TCPKeepAlive is spoofable. The server alive mechanism is valuable when the client or server depend on knowing when a connection has become inactive. The default value is 3. If, for example, ServerAliveInterval (see below) is set to 15 and ServerAliveCountMax is left at the default, if the server becomes unresponsive, ssh will disconnect after approximately 45 seconds. This option applies to protocol version 2 only.
ServerAliveInterval

Sets a timeout interval in seconds after which if no data has been received from the server, ssh(1) will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server. This option applies to protocol version 2 only.

Adobe Flash is back for amd64!

2010-09-17T11:01:00.000-07:00

Finally the 64 bit edition of Flash is somewhat available again! It's called Flash "Square". I tried it last night on my dual core Atom HTPC running Debian Squeeze. It was amazing. Full screen HD YouTube videos play smooth as butter. I've never seen this with Flash in Linux! Amazing!

By the way I still hate Flash but you gotta live with it so...

See my edits in the Debian wiki on how to install on a 64 bit platform:
http://wiki.debian.org/FlashPlayer#Debian5.0.27Lenny.27and.27Squeeze.27amd64

d.o.t.s in your Gmail address demystified

2010-09-17T10:00:00.000-07:00

This is probably not a surpsrise to anyone but me but I still want to share.

According to Google:

homerjsimpson@gmail.com = hom.er.j.sim.ps.on@gmail.com
homerjsimpson@gmail.com = HOMERJSIMPSON@gmail.com
homerjsimpson@gmail.com = Homer.J.Simpson@gmail.com

Sweet! Now to make it even harder to contact me just e-mail me at e(dot)r(dot)n(dot)e(dot)s(dot)t(dot)o(dot)o(dot)n(dot)g(dot)a(dot)r(dot)o AT gmail(dot)com

http://mail.google.com/support/bin/answer.py?hl=en&ctx=mail&answer=10313#

1,741 times faster than dial up!

2010-08-30T20:43:00.000-07:00

I am checking out the grand new offices for the RAMCloud project at Stanford. The internet here is 1,741 times faster than 56k dial-up! Ironically storage speeds have not grown at the same rate as Internet speeds, that's what RAMCloud hopes to solve (in the data center).

This is insane bandwidth. Oh and every PC here gets a public IP! I'm in Internet heaven.

Open Government Data and JasperReports

2010-08-17T23:46:00.000-07:00

Major geek out this evening. I was supposed to go to the gym with Diego but he passed out on the couch from jet lag and I decided to hit the open source reporting space instead (the obvious choice!). I'm evaluating JasperReports, an open source reporting engine. I'm very impressed with the results.

Installation:
Installation was a breeze - I fired up a Debian 5 VPS on my home server and after downloading the JasperServer package I had my TomCat, MySQL, Java installs all done for me. I had the option to configure each component separately - I opted to have Jasper do all that for me this time. I also decided to give JasperReports-Pro a try under the time trial demo. There's also the completely free open-source edition.

Data Source:
Once I was able to log into the web interface I had to find some data! I decided to try data.gov. I found some "raw data" on 2008 adoptions per foreign country for United States families. The data came in the form of an .xls file that I cleaned up, turned into a .csv and imported into a separate MySQL server that I had running.

Configuring JasperReports:
Configuring JasperReports was not too bad. There's definitely a learning curve but within a couple of hours I was able to get everything plugged in nicely and a report generated. The basic steps were:

1) Create a JDBC connection to your database (in this case it was another host on my LAN with a mysql server listening for connections). Jasper calls this a JBDC Data Source

JasperReports: Turning Data into Information

2) Create an abstraction of the Data Source and the queries. This can get extremely complicated - you can pull in several tables and several data sources, manipulate them in any way you like (yes, graphically if you like) and then you should have what Jasper calls a Domain. My Domain was very simple, just pulls in the tables from my data source. In the real world this step would be done by an experienced database administrator.

3) Finally you can then set up a report using the data from the Domain created in step two. The idea is that your end users create reports using the JasperServer GUI, the end user really doesn't need to worry about the JBDC data source.

The Results:
After all this work you have a report. A sweet report that is linked to your data source. If I upload data into my database the reports are updated instantly. This is very cool. Since I already have all the data from my table in my Domain, creating other reports and mash-ups is simple.

The "real-world" application doesn't seem obvious in my example here - if you work in an enterprise where data is needed on a constant basis then JasperReports is for you. Reports can be ran on demand by end-users or scheduled and e-mailed on a recurring basis. Here's what my report ended up looking like. It's super basic and just tells me that Guatemalans were the child of choice for 2008.

If you put together all the information available from the government on sites like data.gov and combine it with the power of a tool like JasperReports, having incredibly useful information at your fingertips becomes a reality.

Dropbox in Debian Squeeze

2010-07-21T12:31:00.000-07:00

I decided to give Debian Lenny +1 distribution, Squeeze a try on my work desktop. Things look really good and easy for one of my favorite cloud services, Dropbox. To install (instead of the old manual methods) just enable the non-free repository in your /etc/apt/sources.list:

deb http://mirrors.kernel.org/debian/ squeeze main contrib non-free

Then you can simply install Dropbox with:

apt-get install dropbox

Finally a fix for home, end, page up and page down keys in Mac OS Terminal!

2010-07-12T20:33:00.001-07:00

Thanks Mac Improved Blog man:

http://macimproved.wordpress.com/2010/01/04/fix-page-updown-home-end-in-terminal/

lighttpd: whitelist some IPs while authenticating the rest

2010-07-09T16:51:00.000-07:00

Here's the scenario: you have an office full of people that need access to a certain web app. Some of them probably have insecure passwords and you're too busy to worry about the latest security holes in your web-app. Slow down attackers by allowing your office IP addresses in while denying the open web access until they put in a simple group password.

In other words, this post walks you through having lighttpd allow some IP addresses in (and authenticating with your web app) and others to have to authenticate with mod_auth first, then the web app.

These instructions were tested on Debian Lenny:

First Enable the authentication module:
lighttpd-enable-mod auth
Create the password file, the format is username:password
vim /etc/lighttpd.user
Make the password file owned by the webserver user:
chown www-data:www-data /etc/lighttpd.user
Configure the auth module:
vim /etc/lighttpd/conf-enabled/05-auth.conf
* Comment out the auth.backend = "plain" line
* Comment out the auth.backend.plain.userfile = .... line
* Change the auth.backend.plain.userfile file to the one you created above, /etc/lighttpd.user
Finally, have all IPs authenticated, except for the IP1 and IP2 (add more separated by pipes) by adding the following to /etc/lighttpd/lighttpd.conf:
$HTTP["remoteip"] !~ "IP1|IP2" { auth.require = ( "" => ( "method" => "basic", "realm" => "Employees Only!", "require" => "user=username" ) ) }
Note: you can't use hostnames, only IPs
Reload lighttpd and you're done!

Pico Projectors

2010-05-10T10:15:00.000-07:00

Is the next big (little) thing Pico Projectors? Check out my friend's site http://picoprojector.org/ - he's got pretty comprehensive reviews and news about the miniaturized projectors.

Blackberry Bold 9700 and Mac OS X 10.6.2 Bluetooth Tether for AT&T

2010-03-23T22:05:00.000-07:00

This is a fairly lazy post. I'm assuming you have the devices tethered via Bluetooth - that's fairly common knowledge. What's uncommon is getting it to work. I tried the Blackberry Modem profiles that people cooked up with no luck. What ended up working was this Option N.V device profile. Just follow the screenshots.

* Note password is CINGULAR1

The speeds are under a Mbit up and down. Not great but this works. If this isn't working I suggest you tail -f /var/log/system.log while "Use verbose logging" is checked in the PPP properties.

Source here (obscure CrackBerry post)

Troubleshooting Cisco/Linksys SPA-942 phones with a syslog Server

2010-02-25T12:04:00.000-08:00

Chances are you're using an Asterisk server on Linux with your SPA-941, SPA-942, or SPA-962 VoIP phones. If you're having an issue, like trouble upgrading firmware it's nice to know what the phone is saying. Simple.

Enable remote syslog logging:
On your Linux machine, in this case CentOS, enable remote syslog logging in /etc/sysconfig/syslog by adding a -r to the SYSLOGD_OPTIONS line. It will end up looking something like this:
SYSLOGD_OPTIONS="-r -m 0"

Restart syslog (

/etc/init.d/syslog restart) and then onto the phone.

Enable Logging on the SPA device
On your SPA device under (Admin -> System) enter the IP of your syslog server into the boxes and set a debug level (0 is off, 3 is most verbose):

Look at your logs:
In /var/log/messages you should see some output the phone is putting out. This is an example of a successful firmware upgrade on an SPA-941

Feb 25 13:06:18 10.0.22.114 SPA-941 00:0e:08:23:15:2c -- Requesting upgrade http://10.0.22.15:80/spa941-5-1-8.bin
Feb 25 13:06:19 10.0.22.114 fprv_upgrade_from_buffer_priv(int manual:0, unsigned char *buf: kOsMoS9 fIrMwArE, int file_len: 757258)
Feb 25 13:06:19 10.0.22.114 fprv_upgrade_from_buffer_priv(int manual:0, unsigned char *buf: kOsMoS9 fIrMwArE, int file_len: 757258)
Feb 25 13:06:19 10.0.22.114 ** old magic: kOsMoS9 fIrMwArE, new magic: kOsMoS9 fIrMwArE
Feb 25 13:06:19 10.0.22.114 ** old magic: kOsMoS9 fIrMwArE, new magic: kOsMoS9 fIrMwArE
Feb 25 13:06:44 10.0.22.114 SPA-941 00:0e:08:23:15:2c -- Successful upgrade http://10.0.22.15:80/spa941-5-1-8.bin -- new image 5.1.8

There's not a lot of documentation from Cisco/Linksys on error codes and reboot reasons..on your own there!

Batch convert Asterisk GSM WAV files to mp3

2010-02-16T14:16:00.000-08:00

I am working on a fun project at work to provide web based visual voice-mail for a ticketing system. I realized Flash audio players are not able to play WAV files so had to work around that. Since users are also using voicemail files in other ways I can't just change the output format from Asterisk. The type of file that I'm working with is identifed with file ms0012.WAV as: msg0012.WAV: RIFF (little-endian) data, WAVE audio, GSM 6.10, mono 8000 Hz

First I'll say that I found this post http://www.thiscoolsite.com/?p=73 but like some of the commenters I couldn't get the script to work. The author assumes some other format than Asterisk spits out by default. Lame would complain that Unsupported data format: 0x0031

The Tools
Sox: http://sox.sourceforge.net/
Lame: http://lame.sourceforge.net/

So here's what I do:
Convert GSM encoded wav to Microsoft PCM
sox msg0012.WAV -s msg0012.wav

Convert the PCM wav to mp3:
lame msg0012.wav msg0012.mp3

And here's a script to do this as a batch:

Code snippet - Convert Asterisk GSM encoded WAV files to MP3 on Snipplr

Delete multiple files in Subversion (Linux)

2010-02-02T17:35:00.000-08:00

Removed a ton of files from an SVN repo? This will delete all the files that show up with a ! when you do svn status.

svn rm $( svn status | sed -e '/^!/!d' -e 's/^!//' )

Thanks to Snippler for the snippet.

Official Google Reader Blog: Follow changes to any website

2010-01-25T15:51:00.000-08:00

Official Google Reader Blog: Follow changes to any website

Add sbin to user PATH in Debian

2010-01-18T10:49:00.003-08:00

Debian does not add sbin to your path by default. I have no idea why. I know sbin is supposed to be administrative tools that you would only want to run as root, however, there are some useful ones that don't require root, and even if they need root, why hide them from useful tools like auto complete?

Apparentely it's not a new disucssion - this Debian mailing list thread dates back to a decade ago!

Regardless, I want it in my path - the easiest way that I've found is to modify the top of /etc/profile to remove the if statement that sets path if you're user id 0 (root) or not:

Before:


   if [ "`id -u`" -eq 0 ]; then
     PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
   else
     PATH="/usr/local/bin:/usr/bin:/bin:/usr/games"
  fi

After:


  PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

Save the file, log out and log back in and you're all set. Note that this affects all users on your system.

Modifying Headers for testing Virtual Hosting

2010-01-04T09:21:00.006-08:00

While working on a website hosted on lighty's virtual hosting environment I realized I wanted a natural way to test the site before pointing the DNS over. If you're sharing one IP for multiple sites then this tip is useful. It uses a Firefox add-on.

Get the "Modify Headers" Firefox add-on here.

Once installed go to Tools -> Modify Headers

Action -> Modify

Name -> Host

Value -> Domain you want to "spoof"

See a list of other headers you can play with here.

Type accents and other Latin American punctuation on your Mac

2010-01-03T14:21:00.012-08:00

I'm helping type some of my grandfather's writing from the 1960s (written on a typewriter, not even worth trying OCR) and I needed some punctuation love on my MacBook, this is what I found.

Accents:
To write accents on letters, just tap Option (alt) + e and then the letter:

á	option + e	then a
é	option + e	then e
í	option + e	then i
ó	option + e	then o
ú	option + e	then u

Enyay (ñ)
For ñ, tap Option (alt) + n and then n.

Punctuation:

¡	option + 1
¿	option + shift + ?

Love Google Calendar!

2009-11-23T10:08:00.002-08:00

I always use the quick add feature of Google Calendar to add events, that way I can type in things like "Call with Jack Tuesday morning at 9" and an event is added. Today I figured I'd paste this directly from an email:

Blake has a meeting in Houston on Tuesday. We need to change the staff
meeting to Wednesday. I was thinking 10:00 if that works for you.

It seemed like a long shot, but it worked! It made an event on Wednesday at 10 am, I was surprised and then realized that in the future these sorts of things will be parsed from your email automatically and you'll simply accept them into your calendar. I like!

Nagios Asterisk IAX2 POKE with check_asterisk.pl

2009-11-01T13:35:00.004-08:00

It seems that since around Asterisk 1.4.26.2 the Nagios contrib plugin check_asterisk has stopped working for the "iax ping" feature. I turned on iax debugging in Asterisk and did a packet capture. Seems like the only reason why it worked was because it sent a 0x06 REJECT frame and Asterisk would reply with invalid. I changed it to send a 0x1e POKE request and submitted a small patch to fix this in the plugin. I seriously doubt the Nagios Plugins team will do anything with it since there's no upstream maintainer and it's in the contrib directory so here's the patch:


--- check_asterisk.pl 2009-09-24 03:13:56.000000000 -0700
+++ check_asterisk.pl.changed 2009-11-01 13:10:06.000000000 -0800
@@ -31,7 +31,8 @@
 my $iax_timestamp = "00000000"; 
 my $iax_outbound_seq = "00"; 
 my $iax_inbound_seq = "00"; 
-my $iax_type = "06"; #IAX_Control 
+my $iax_type = "06"; #IAX_Control
+my $iax_frame_value = "1e"; #POKE
 
 sub ok {
  $s = shift;
@@ -224,7 +225,7 @@
  socket(PING, PF_INET, SOCK_DGRAM, getprotobyname("udp")); 
 
  $msg = pack "H24", $iax_src_call . $iax_dst_call . $iax_timestamp .
-  $iax_outbound_seq . $iax_inbound_seq . $iax_type . $iax_type;
+  $iax_outbound_seq . $iax_inbound_seq . $iax_type . $iax_frame_value ;
 
  $ipaddr = inet_aton($host); 
  $sendto = sockaddr_in($port,$ipaddr);

You should also check out this patch that adds performance data to the Nagios plugin. Good for graphing channel usage over time.

References:
* IAX2 RFC Document
* Nagios Plugins
* Bug/Patch Report

Linux Software RAID Nagios Monitoring & Failure Simulation

2009-10-12T20:39:00.013-07:00

It's always nice to monitor to make sure your technology is working properly, it's nicer knowing that when it fails it will alert you to what happened so sometimes you need to test the scenarios. This is just an overview using Debian Lenny of how to use the Linux mdadm tool to create, destroy and rebuild a software RAID 1 (mirror) device. Throughout the process you will see the effects of the Nagios check_linux_raid plugin from the Nagios Plugins project.

First I installed the Nagios plugins and mdadm:

>apt-get install mdadm nagios-plugins

I used fdisk to create two partitions: /dev/sdc1 & /dev/sdc2 (of equal size, in this case 40 MB, I know, I'm generous.)

I then used the following command to create a RAID 1 array with the two disks. I happened to call it md0:

>mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdc2

Created an ext3 filesystem on my array:

>mkfs.ext3 /dev/md0

Mounted the new file system in an arbitrary location:

>mount /dev/md0 /media/raid

The array is now active and happy, here's how to verify:

>mdadm --detail /dev/md0
/dev/md0:
...
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 34 1 active sync /dev/sdc2

Now I will run the first test of the Nagios plugin, you can give it the device name if you like or it can detect your devices for you. As you can see all OK (exit status 0):

>/usr/lib/nagios/plugins/check_linux_raid
OK md0 status=[UU].

Now I'm going to set one of my block devices as faulty:

>mdadm /dev/md0 -f /dev/sdc2
mdadm: set /dev/sdc2 faulty in /dev/md0

To verify the array is degraded we'll look at the details again:

>mdadm --detail /dev/md0
/dev/md0:
...
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0

UUID : 3103faf6:34bec8ae:2689ae2c:f32f1e02 (local to host buenos-aires)
Events : 0.8

Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 0 0 1 removed

2 8 34 - faulty spare /dev/sdc2

You'll notice /dev/sdc2 is now marked as a faulty spare.

Another check of the Nagios Plugin reveals an exit code of 2 and the service as critical:

>/usr/lib/nagios/plugins/check_linux_raid
CRITICAL md0 status=[U_].

I now remove the faulty device from the array completely (note it must be marked faulty first per the last step):

>mdadm /dev/md0 -r /dev/sdc2
mdadm: hot removed /dev/sdc2

Again test the plugin, still marked as down:

>./check_linux_raid
CRITICAL md0 status=[U_].

Now I can re-add the device to the array (since there's really nothing wrong with it). Once it rebuilds you can test again and it will show as OK on the Nagios plugin.

mdadm /dev/md0 -a /dev/sdc2

You can read more about Linux software RAID here. Or about Nagios here.

Moving, Migrating, or Backing Up Microsoft PPTP VPN Profiles

2009-08-05T13:43:00.006-07:00

So I'm moving from one Windows XP machine to another and I don't want to have to set up all the VPN profiles again. They are stored in the file

C:\Documents and Settings\All
Users\Application Data\Microsoft\Network\Connections\Pbk\RASPhone.pbk

Just make a copy of that file and put it on your new system. It doesn't save your passwords though. I'll update this post if I find a way to back up the saved passwords.

Comparison of Windows Mobile 6.1 and 6.5

2009-07-16T22:25:00.000-07:00

Complaints about Windows Mobile 6.1
I've owned an HTC Fuze (aka Touch Pro or P4600) Windows Mobile phone since mid February of 2009. General impressions of the hardware are good, a solid construction paired with a beautiful VGA resolution screen. My first PC came with a 15 inch 640x480 monitor, this has a 2.8 inch 480x640, 3:4 aspect ration screen. 5X smaller, with 16 bit color. This is the most attractive thing about this device. Dead sexy.

So far I've been rather unimpressed by the lack of stability on the phone. There are many quirks to it. One especially annoying one is when you start to run low on batteries the phones' phony notification system turns on, along with the back light and radios to alert you that the battery is dying. If you didn't notice, it's kind enough to stay on until it goes completely dead. This is a major issue with the phone, it already has pretty lousy battery life (though not too far from what the iPhone gets).

Another major annoyance is that the keyboard stops working when you are typing sometimes. I assume there's a background process that eats all the CPU cycles. There's little consistency to the problem in that sometimes input will be queued and released all at one or not at all. Pain in the butt to type on when this happens, the keyboard on the Fuze is one of the best I've seen - seems like the software is to blame here.

The phone needs a lot of rebooting, especially using programs other than email. It is so bad that I read on a forum somewhere "Do this procedure every reboot or once a day" - Like rebooting once a day is totally normal - that's the Windows Me mentality that has held a dark shitty cloud over Microsoft for some time.

Windows Mobile 6.5
Windows Mobile 6.5 is set to be seen on the first consumer device by September 2009, most phone manufacturers received the software in May. I'm using a beta preview.

One of the problems with 6.1 is that I have to run a separate package on top of Windows called Touch Flo 3D. This makes the phone a bit more touch friendly but adds a giant layer of clunk. Below TF3D is 6.1 so you only get the nicer experience some of the time. With 6.5 one of the major changes is a redesign of the Today screen. It gives you quicker finger-friendly access to email, text, music, photos, videos, and bookmarks. Though TF3D also does this it's a memory hungry app that resides on top of the OS, this is better integrated and lighter.

The second major area of improvement is the Start Menu. The start menu uses hexagonal shapes for icon placement. The icons are a beautiful resolution and quite easy to navigate with the fingers, stylus, . Scrolling is pretty nice and you can re-arrange the icons by sending them to the top.

Windows Mobile was designed a LONG time ago and I don't think anyone thought that it would be ran on a mobile phone with a touch screen. It is really designed for a stylus - not a finger. I find myself when doing heavy computing on the device with the keyboard slid out, the stylus in my hand and my thumbs on the screen. Beleive it or not - this is actually a really awesome way to use the phone - I can be doing quite a bit at once. To conclude this statement - 6.5 adds some finger friendly features however it does not stop you from having to use the corner of a finger nail to press the dreaded OK button. I often open up the keyboard and press the OK button there instead of messing with it.

Overall - I'm pretty happy with the stability that 6.5 offers. There's a Marketplace feature (similar to iPhone's App Store) that is coming with the release. That should be interesting.

Naming of ethernet devices on Debian

2009-04-12T08:19:00.003-07:00

If you change network cards in Debian, the new card will get a new "ethX" name, assuming that perhaps you want to put the old one back one day and keep the configuration.

Sometimes this isn't the case and you don't want to have ethernet devices named eth5 or similar.

A simple fix, the file:

/etc/udev/rules.d/70-persistent-net.rules

has the rules to persistently name your ethernet interfaces, simply edit that and remove any entries that are no longer needed; renaming the other ones to your needs. The file had the driver and MAC address so they are simple to find. Here's an example (with the macs slightly changed):


# PCI device 0x8086:0x1076 (e1000)
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="00:14:22:72:d7:2d", NAME="eth0"

# PCI device 0x8086:0x1076 (e1000)
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="00:14:22:72:d7:2c", NAME="eth1"

# USB device 0b95:7720 (asix)
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="00:50:5b:04:85:5f", NAME="eth2"

Pasting in Vim

2009-03-25T14:51:00.005-07:00

For years now I've been using nano with the no word wrap flag to paste stuff into *nix terminals. This kind of sucks because Vim is my favorite editor, however when you try to paste it indents everything for you and ends up with a bloody mess. Well I found that you can:

:set paste

paste what you want and then

:set nopaste when you're done. Wooohoo!

Update on my eeePC

2009-03-02T13:36:00.005-08:00

So a few weeks ago I was working on my eeePC and got some kernel panic warnings and disk read errors. I rebooted to run a full fsck..Sure enough my drive was completely corrupt. I reformatted after playing around with badblocks. Still no joy after that, the drive had a physical problem with it.

I started the somewhat tedious process of sending the netbook in for warranty repairs. I must say that Asus obfuscates the whole process, most likely in hopes that people will simply give up and buy a new laptop. The phone menus were terrible, only after emailing them they told me the right key sequence to push to successfully navigate to the warranty section. The key sequence would have been impossible to get to since it was not advertised by their phone menu. As a guy that occasionally programs auto attendant menus - that's terrible. Fsck that.

The process took about 2 weeks but I got back Friday.

I decided to give openbox + lxde a try, running on the newly released Debian Lenny. So far so good. I ran into an issue with getting wireless going on Lenny final, I had to compile the madwifi source since the open source driver, ath5k did not do what it advertised. I'm not a huge GNU purist so I don't really care.

I'm disappointed to have hardware problems so early in this machine's life..might be just a fluke though.

Power consumption of my Windows Desktop

2008-11-14T06:59:00.004-08:00

I ran into a rather interesting discovery regarding power consumption of my PC while in sleep mode.

Using a Kill-A-Watt I measured how much power my computer uses.

While under normal operation, I use around 120-170 Watts. This is reasonable for my hardware - Core 2 Duo 7200, GeForce 6800GT, 22 inch ViewSonic flat panel.

Then I tested the watt usage while the machine was in a "Sleep" state. The result was a surprisingly low 13 watts.

I noticed that I could not wake my computer up using my USB keyboard, only the power button. So in Control Panel -> System -> Hardware -> Device Manager -> Keyboards -> Power Management I checked "Allow this device to bring the computer out of standby". I then re-ran the test with the Kill-A-Watt.

With this option enabled my consumption during sleep mode went up to 66 watts! That's a 500% increase!

I leave my PC on 24x7x365. Looking at my last bill I pay .16 cents for a kilowatt. Having this option turned on will cost me an extra $75/ year. No thanks, I'll just push the power button to wake up from now on.

If anyone else has a way to measure energy consumption - post your results.

Two Months with my Asus eeePC

2008-10-20T19:39:00.007-07:00

Back in August I replaced my aging 15.4 inch Sony laptop with an Asus eeePC 900. This is somewhat of a review of my experience so far; wrestling with various operating systems and my impressions of the hardware.

Specs
The particular eeePC 900 I purchased from Newegg has the following specs:

Celeron M processor clocked @ 900Mhz
4 GB Solid State Drive for OS
16GB Solid State Drive for data
Xandros Linux
1 GB DDR2 RAM
1024x600 8.9-inch screen
webcam/microphone/vga out/sd reader

Operating Systems
First try...Egg
I decided to give the pre-loaded, linux based, Xandros OS a chance. The very first impression I had of it was quite positive. It booted EXTREMELY fast. In fact, the fastest boot of any operating system I have ever seen; 20 seconds. Once I logged in I quickly found my way around the simple tab based interface. It reminded me a little bit of Geo Works (an operating system that was on my first computer; a Laser 386.) I did my best to connect in my work email, extend firefox, customize it a bit and get some use out of it. I quickly became frustrated with the disappointing options for adding software and customizing the existing software. Besides the quick boot times - I really did not find anything good about the operating system that came on it. I didn't expect that I would like it; I simply wanted to give it a chance.

Correction - one more almost cool feature was the VoiceCommand application - though I found myself talking to my PC too much and felt a bit odd.

Second go Around - Caterpillar
After a week or so of Xandros - I decided to load Ubuntu 8.04 onto the eee PC. Ubuntu has been a solid performer for me on the desktop. It's a full featured operating system and comes with everything you need out of the box to be productive- this I will not deny it.
I easily loaded a live CD onto a flash drive and installed from there. Install went smooth and I think almost everything worked out of the box after a few minutes of applying some quick patches.
However, Ubuntu has become a very FAT distribution. Package dependencies are a bit of a nightmare. On my little eee PC this was simply too much. Boot times were disappointing and trying to remove extra packages was a nightmare with Ubuntu's heavy dependencies on tools that make the average Linux user's life easy. Those tools I started to hate more and more. Then I ran into some trouble getting my microphone to work and in a moment of desperation, I decided to try another operating system.

Third experimentation - Pupa
When I gave up on Ubuntu for being too fat I decided to give Windows XP SP3 a chance. For this, unfortunately, I was forced to buy an external CD-ROM drive. I ran into some trouble getting the eee PC to boot into an image of the Windows installer on a thumb drive. No big deal, I needed one anyways.
I decided to go with a slimmed down image of XP SP3 using XP-Lite. The install went smooth. Of course, nothing worked out of the box (networking, acpi, etc). Asus was nice enough to include a CD of the Windows drivers necessary to run on this machine. The driver install process took up almost 1.5 hours of my time. Once I got past that, the computer ran pretty well with XP. It booted quickly, seemed pretty stable, etc. I used it with XP for close to a month. One day I decided to tether my Blackberry Perl with the laptop to receive internet wirelessly. I loaded the huge program Blackberry Manager and proceeded with a sync. Failed. Many many failures and re-installations. The software slowed my computer to a crawl. It was pathetic. Then I was reminded why I don't use Windows - it gets way too slow once you add software. After some more disappointments - and lack of the basic Linux networking tools that I have grown to need, I decided once again to switch.

Fourth Iteration - Butterfly
For the final chapter of my eee PC, which I named Wall-eee after Pixar's latest masterpiece - Wall-E I decided to go with Debian Lenny. Lenny is Debian's upcoming release and is still considered "testing". Installation was not difficult since I already had an external CD drive. Out of the box, wired and wireless networking did not work. Wired networking was easy, wireless a bit more work but simple using module-assistant. I decided to go with xfce4 as my desktop enviornment. Finally, I felt right at home. I was in full control of the packages that were being installed - nothing extra, nothing fancy. Some of the applications I use on Debian are:
* xfce4-places plugin - quick access to folders, documents, and removable media (writen by my brother Diego)
* date-time plugin - a panel plugin that gives you better control of the (you guessed it!) date and time and shows a calendar - again partial to this because my brother maintains it.
* finch - an ncurses based front end to libpurple (pidgin). It's very lightweight and allows me to connect into AIM, Gtalk, MSN and others. It has full support for sending/receiving files - just console based so it's not "pretty".
* Unison - a 2 way file synchronizer that works over SSH using rsync - very useful for work.

Although configuration of Debain can be quite a task - I feel security in it's stableness and maturity. I have faith that when I change something it will stay the way I left it. In Ubuntu - there is little guarantee of that since they attempt to have everything pre-configured and re-configured to suit their taste.
With Debian, my eee PC boots in around 45 seconds. I improved on this by trimming some fat from boot up services, removing extra boot time packages - like some loggers and not using GDM as a start up manager (just log in and run startx). Next I will compile my own kernel with all needed packages compiled in, I will follow up what my boot time is after that.

Hardware
So far I have focused on the software with little mention of the hardware. I am absolutely in love with this piece of hardware. It's exteremely solid. The only moving part that I am aware of is the CPU fan, which seldom runs. The screen passes my hing test (I can pick the laptop up by using the screen and the screen hinge does not move). The camera and microphone get some good use using Skype or a softphone. Plenty of USB ports (they saved my sanity during Hurricane Ike by charging 2 iPods and my phone). The power adapter is very small. The machine weighs close to nothing. I carry it around everywhere. It's small size comes in handy when working on a 66 block or something where I'm referring to documentation (holding Wall-eee) and using a tool with the other hand. Battery life is comparable to my Sony - around 1.5 to 2.5 hours. Not impressive but not bad either. I'm sure I could squeeze more out of it by tweaking with the screen brightness, turning off wifi, throttling the CPU, etc. The keyboard is pretty easy to get use to - not full size by any means but I can type on it with relative ease.

In the end, I would recommend this PC to anyone who needs a super portable PC. If you have to do any extended work on a laptop - this thing is not for you. The screen real estate is lacking. If you are constantly moving around, booting up, changing network configs, running diagnostics, working in a cramped server closet - this thing is your best friend.

Wireless Antenna Experimentation and Review

2008-07-22T20:42:00.014-07:00

I purchased two Gigaware 21-162 wireless antennas from Radio Shack in order to get a longer range wireless connection between two routers. I am going to move into an apartment across the street and from a friend who already pays for internet access and is happy to share it with me. I am going to replace the stock antennas on my 2 Linksys WRT54G routers with the ones from Gigaware in hopes to get a clear signal.

The firmware on the routers I am using are open source router distros that will allow me more control of the transmitters and allow the Linksys WRT54G routers to be put into a bridged mode (which allows for one to connect to the other and bridge two networks into one wirelessly separated logical network ). The master router uses DD-WRT v24 RC6 and the slave one uses Tomato Firmware v1.19.1463

You can relatively easily achieve the router configuration portion of this with these instructions from the DD-WRT Wiki. The Tomato config is just as simple, the GUIs are very intuitive.

The antennas are very easy to install, I unscrewed the factory antennas and screwed the replacements right in. The antennas are twice as long as the factory antennas and give my WRT54Gs a nice ego boost.

The antennas claim to have a 7 dBi gain. I wanted to see what sort of gain I would get after installing them on a router running DD-WRT. I left the advanced wireless settings at their default and got these results:
The results come from running the command: watch "iwlist wlan0 scan" | head on my Debian laptop, which allow me to see a "Quality" measurement and a "Signal Level" measurement. The laptop was about 20 feet from the router and did not move during the tests. Some of the quality measurements would fluctuate by around 4 points, the figures where calculated using an average. (Note that Signal Levels closer to zero are better).

To interpret that data a little bit, installing the antennas gave me a 15% boost if measured by "Signal Levels" and a 12.5% increase if measured in "Quality". In dB the difference was 6 dB, the box claims dBi. (Not sure what the difference is between dB and dBi.

Yes, it would be nice to know how these results affect coverage area...I have not performed any long distance tests yet as that requires my neighbor's to know that I am a complete geek. Don't worry they will come. Additionally, I may add that there are surely some problems with this data - don't quote me on these numbers.

Specifications of the Gigaware 25-162 (from the box):

Purchased from: Radio Shack
Purchase Price: $7.97
Frequency: 2.4 GHz
Gain: 7dBi
Transmission pattern, 20 degrees vertical
VSWR: 2:1 max
Impedence: 50 ohms
Power handling: 2 watts

I will be moving September 1st and will post the results of the internet sharing attempt.

Attachments in MoinMoin 1.6

2008-05-07T23:02:00.002-07:00

Alternate Attachment Syntax in MoinMoin 1.6

As of MoinMoin wiki version 1.6 (or so) attaching files to a page has changed the action that previous versions performed. When a file is attached with the format {{attachment:foo.pdf}} the file is not available for direct download. Instead, the user is sent to an intermediate page where he/she can manage the attachments, including downloading it. Sometimes I prefer the old behavior, most notably when I want to make it easier for the reader.

The format to use is:

[[attachment:foo.pdf|description|&do=get]]

where foo.pdf is the file you want to upload and description is a short name for the file.

Thanks to Tre56k in the MoinMoin IRC channel for this tip.

POP3 Commands

2008-05-07T22:37:00.006-07:00

For use when troubleshooting email problems with telnet (or parsing email in your head) .

First you must telnet to the POP3 sever, in Windows or Linux, enter a terminal and type:

telnet [host [port]]

Where host is the POP3 server you want to connect to and port is generally the standard POP3 port 110.

Minimal POP3 Commands:
- USER name valid in the AUTHORIZATION state
- PASS string
- QUIT
- STAT valid in the TRANSACTION state
- LIST [msg]
- RETR msg
- DELE msg
- NOOP
- RSET
- QUIT valid in the UPDATE state
Optional POP3 Commands:
- APOP name digest valid in the AUTHORIZATION state
- TOP msg n valid in the TRANSACTION state
- UIDL [msg]
POP3 Replies:
- +OK
- -ERR

This information gathered via various websites and this document: ftp://ftp.isi.edu/in-notes/rfc1939.txt