Eschew Obfuscation: Category Tech Tips

MythTV Trial

Max Dunn — Mon, 21 Jan 2008 09:15:00 -0800

Several weeks ago, my Windows Media Center started having problems. TV playback would stutter and the CPU was pegged at 100%. After looking for solutions on the web, I deinstalled all codecs and then reinstalled NVidia PureVideo. Now TV and DVD’s wouldn’t play at all! Since I was tired of continually hassling with Media Center, I decided to finally give the Linux based MythTV a try.

Avoid Spam - Don't Use Simple Email Addresses

Max Dunn — Fri, 02 Nov 2007 07:08:00 -0800

When choosing an email address, it is often tempting to use just your first name or last name like lisa@mycompany.com or smith@mycompany.com. Don’t do it!

The problem with these simple email addresses is that spammers often use directory harvest attacks to find new email addresses to send spam to. In these attacks, they try using all sorts of different email names to see if any go through. Here is an edited portion of my mail log file that records these attacks:

...
Oct 28 01:51:26 <barker@testcompany.com>... User unknown
Oct 28 01:51:27 <barnes@testcompany.com>... User unknown
Oct 28 01:51:28 <barnett@testcompany.com>... User unknown
Oct 28 01:51:29 <barrett@testcompany.com>... User unknown
Oct 28 01:51:30 <bates@testcompany.com>... User unknown
...

Designing Web Sites That Look Good Everywhere

Max Dunn — Tue, 11 Sep 2007 13:25:00 -0700

There are three main issues to consider when designing web sites that will look good to the widest number of visitors: monitor size, OS/browser and colors.

Monitor Size

First, no-one uses 640×480 anymore, so this should not be considered. Next up only about 8% of users have 800×600 displays. The biggest group is 1024×768 which accounts for about 50% of user’s displays. At the 1280×1024, we have about 17% of the users. The rest, about 25% have higher resolution displays.

What this means is that to design for the widest group, web pages should be displayable on a 800×600 display, but can be optimized for best viewing on a 1024×768 display.

Keep in mind though, when thinking about the design is that it is hard to read long lines of text and that narrower web pages are often easier to read. Also, when users scan web pages, they look in an F-shaped pattern: first along the top and then the left side. So having a wide page with a lot of important information on the right side will likely not be effective.

Initial Subversion (SVN) import

Max Dunn — Sat, 28 Apr 2007 09:31:00 -0700

If you are using the SVN command line, here is a trick that makes it easier to import your files into Subversion. The problem with the usual method is that after importing your files, you have to check them out to a different directory, rather than continuing to work with them in the same location. This methods avoids that problem:

Automatically starting Apache with chkconfig

Max Dunn — Tue, 20 Mar 2007 07:44:00 -0800

It is interesting that Apache 2.2 doesn’t include a startup script that is compatible with the Linux ‘chkconfig’ command. Here is how you fix this:

First, add this information to the top of /usr/local/apache2/bin/apachectl:

# chkconfig: - 85 15
# description: Apache is a World Wide Web server.  \
#              It is used to serve HTML files and CGI.

Then change to /etc/init.d and do:

sudo ln -s /usr/local/apache2/bin/apachectl httpd
sudo /sbin/chkconfig --add httpd
sudo /sbin/chkconfig --level 2345 httpd on

That’s it! Now Apache will start when the computer starts up.

MySQL Recovery

Max Dunn — Mon, 05 Mar 2007 08:57:00 -0800

We had a bad crash yesterday on our MySQL 5.0.22 database. Our guess is that we ran out of hard disk space during a large transaction and the database somehow got corrupted. Worse, restarting MySQL didn’t clear the error, but continued to have the problem. Luckily, we could still read from the database so we were able to do a current backup which allowed us to recreate the database and re-import the data. Here are the steps we used:

In my.conf use:
innodb_force_recovery = 6
(or lower) to bring the database up.
(See http://www.mysql.org/doc/refman/5.1/en/forcing-recovery.html)
Dump the tables. For example. to dump the ‘maxwiki’ table use:
mysqldump - mysqldump --database maxwiki -u (username) -p (password) > maxwiki.sql
Repeat for all other tables
Shut down the database
Move all files out of /usr/bin/mysql and put elsewhere

Re-init with:

  mysqld_safe --user=mysql
  mysql_install_db --user=mysql
  mysqladmin -u (username) password '(password)'

Reload all the tables from the sql files
Reset the users

Setting up Subversion 1.4 with Apache 2

Max Dunn — Tue, 23 Jan 2007 09:50:00 -0800

I like Linux, I really do. That is one reason why I got a Mac last year and also why I host my web sites on Linux servers. However, for all the talk about Microsoft’s DLL Hell, it is nothing compared to Linux Dependency Version Hell. Those of you that have tried installing new versions of software on Linux know what I mean. Whenever you upgrade one piece it causes a cascading effect of having to upgrade lots of other pieces. And sometimes there is a problem with the latest version of a library so you have to use an older version. Now if you have something like apt-get, yum or urpmi, it can help a lot by installing all the needed dependencies for you. But once you venture outside the official versions, it will take a long time to get everything right.

Take for instance, what should be a straightforward task of getting Subversion 1.4 (SVN) working with Apache 2. Of course, you don’t have to have these working together because you can use svnserve by itself. But this only lets you set access control for the whole repository, rather than the finer grained access you get going through http, and you also need to add all svn users to Linux, which is problematic. So to get the advantage of having Subversion working through Apache using http, I spent a day figuring it out.

In the end, the procedure turned out to be fairly straightforward and to save others the pain I went through, here is the trick:

Use the Apache apr and apr-util libraries when making Subversion.

On a standard install, you will probably have libapr-1.so and libaprutil-1.so in both /usr/local/apache2/lib and /usr/local/apr/lib and maybe even /usr/lib. If you configure Subversion with the defaults, it will find the apr libraries in /usr/lib. And if they are not the exact same version as in /usr/local/apache2/lib, there won’t be any errors but when you try to access the repository, you will get error messages like:

Could not fetch resource information.  [500, #0]
Could not open the requested SVN filesystem  [500, #2]
Internal error: Can't open file '/var/svn/format': No such file or directory

So the best thing is to delete all the libapr* files everywhere but /usr/local/apache2/lib and then configure Subversion like this:

./configure --with-apxs=/usr/local/apache2/bin/apxs \
--with-apr=/usr/local/apache2/bin/apr-1-config \
--with-apr-util=/usr/local/apache2/bin/apu-1-config

After that, you should be able to follow the Subversion/Apache setup instructions with no problems.

Differences between RewriteRule in .htaccess and httpd.conf

Max Dunn — Fri, 12 Jan 2007 15:08:00 -0800

This is probably obvious to people who know Apache well, but I just spent a half day struggling with the differences between RewriteRule in .htaccess and httpd.conf. I was assuming they would behave the same way, but there are slight differences:

Differences

httpd.conf

The request URI will have a leading slash
The leading slash needs to remain to find cached files
When using -f or !-f, it needs to have the document root on front

.htaccess

The request URI won’t have a leading slash
Can’t have a leading slash to find cached files
When using -f or !-f, don’t use the document root

Examples

Here is an example that looks in the public/cache directory for cached files:

httpd.conf


RewriteRule ^/$ /cache/index.html [QSA]
RewriteRule ^/([^.]+)$ /cache/$1.html [QSA]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]

.htaccess


RewriteRule ^$ cache/index.html [QSA]
RewriteRule ^([^.]+)$ cache/$1.html [QSA]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]

Rails

Just to complete this example, add this line to a before_filter method in application.rb to have Rails cache files to this directory:

ApplicationController::page_cache_directory = "#{RAILS_ROOT}/public/cache"

Installing MySQL Ruby Gem on Mac OS X 10.4

Max Dunn — Tue, 12 Dec 2006 23:14:00 -0800

There are some problems installing the Ruby Gem MySQL on the Mac 10.4. When you do a:

sudo gem install mysql

you will likely get an error like:


*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

To fix this problem, run this:


sudo gem install mysql -- --with-mysql-dir=/usr/local/mysql

(Solution From How to Install on OSX)

Next, you will probably see this error:

mysql.c: In function 'Init_mysql':
mysql.c:2015: error: 'ulong' undeclared (first use in this function)
...

To fix this, change to /usr/local/lib/ruby/gems/1.8/gems/mysql-2.7 and edit “mysql.c” to add this to the top:


#define ulong unsigned long

(Solution from: I.NFECTIO.US)

Now recompile with:

sudo make
sudo make install
sudo make clean

That should do it!

SVN Connection problem

Max Dunn — Tue, 12 Dec 2006 09:16:00 -0800

We use RubyForge to host MaxWiki. A few days ago, I started having problems connecting to our Subversion (SVN) repository on RubyForge. In Eclipse, I would get the message:

svn: Cannot connect to 'svn+ssh://maxdunn210@rubyforge.org/': 
There was a problem while connecting to rubyforge.org:22

If I tried to use the SVN Repository Explorer in Eclipse, it would report something like:

Cannot find folder "

Running the svn command line would produce:

ssh_exchange_identification: Connection closed by remote host
svn: Connection closed unexpectedly

Since I had been struggling recently with some other SVN problems and had updated Eclipse to use the latest Subclipse version 1.1.9, I thought the problem had something to do with this upgrade. Also Wido had been struggling with problems connecting to the RubyForge SVN repository on his Windows box and finally got everything working fine when he setup a convoluted arrangement to use public key authentication. However, since these instructions didn’t apply to my Mac, and I knew that it worked before, I kept banging my head against the wall trying different things.

Finally this morning, I ran into a note on RubyForge that described this exact error.

It turns out that RubyForge uses a service called DenyHosts to to help thwart SSH server attacks. Somehow my IP got on this list, and that was what was causing this problem!

Since I use DSL with a dynamic IP, it is possible that the person that had this IP before me was trying to hack and got the IP on the blacklist. Or maybe it was just my futzing with the upgrade that triggered this. In any event, I just reset my DSL modem to get a new IP, and – voila – I can access RubyForge SVN again!

Probably if I setup Eclipse and SSH to use public key authentication, I wouldn’t have this problem, but that will have to wait until next time.

"svn: Malformed network data" error in Eclipse

Max Dunn — Thu, 07 Dec 2006 10:37:00 -0800

Several weeks ago, I changed from a Windows laptop to a MacBook. After I got Eclipse setup and went to check-in some code to RubyForge, I received an error:

svn: Commit failed (details follow):
svn: Malformed network data

At first I figured that this was just a problem with the RadRails Subclipse module on my Mac, but then I tried it on my old Windows computer that used to work fine, and got the same error!

After googling around, it seems that this is a known issue when SVN server is updated to version 1.4, but older clients are used. The clients were fixed in October, but there hasn’t been a new official Subclipse build so the automatic Eclipse update doesn’t help. I spent a day trying to figure out how to compile a new Eclipse version, but no luck.

However, there are two workarounds. First, the svn command line works fine.

The second workaround relies on the fact that the changed files are actually committed, but that the error just prevents the local svn repository from recognizing that. So after the error, doing an “update” will merge the changes and everything will be good.

If you have added or deleted files, however, the update trick won’t work. In this case, it is necessary to delete the entire directory that contains the added or deleted files. Be sure to do this outside of Eclipse with Explorer or Finder! Then update the files and it will bring everything down fine.

Versions

Program	Version
RadRails	v.0.7.1
Subclipse	1.0.3
svn command line	1.3.1

Fix

Mark Phippard added a comment explaining the problem. Here is a summary:

The issue is JavaSVN. It had a problem talking to Subversion 1.4.x server. RadRails can be used with Eclipse 3.2 and then you can install Subclipse 1.1.9 which includes SVN 1.4 clients.

So I installed Eclipse 3.2 and RadRails on top. However, when I installed Subclipse according to the instructions:

Eclipse: Help => Software Updates => Find and Install
Search for new features to install
New Remote Site
URL => http://subclipse.tigris.org/update_1.0.x

it gave me version Subclipse 1.0.3. The trick to get Subclipse 1.1.9 was to use the update site:

URL => http://subclipse.tigris.org/update_1.2.x

Apache Redirects to the Rescue

Max Dunn — Sat, 07 Oct 2006 12:36:00 -0700

There are a couple of technical problems with my web site. The first is that to get into this blog, you need to add the trailing slash at the end of “typo” like this:

http://www.maxdunn.com/typo/

Otherwise, the wiki gives a page not found error for the “typo” page. Of course, when most people type in the URL manually, they won’t add the trailing slash and then get confused when it doesn’t pull up my blog.

I have tried various things in the Rails routing.rb file to correct for this, but none worked. However, a simple change to .htaccess did the trick:

RedirectMatch permanent ^/typo$ http://www.maxdunn.com/typo/

Now another problem I have is that my ISP Site5 maps maxdunn.com to www.maxdunn.com and there is no way to turn this off. This causes several problems:

When logged into www.maxdunn.com, you won’t be logged in if you access it as maxdunn.com.
If someone first accesses the site using maxdunn.com, then the page cache will create all links as maxdunn.com and serve these up, even if the page is later accessed as www.maxdunn.com.
If the site is accessed with both maxdunn.com and www.maxdunn.com, then Google and other search engines will think these are two sites and neither will be ranked as highly.

To fix this problem, there is another simple .htaccess addition that will redirect all maxdunn.com requests to www.maxdunn.com:

RewriteCond %{HTTP_HOST} ^maxdunn.com$ [NC]
RewriteRule ^(.*)$ http://www.maxdunn.com/$1 [L,R=301]

Improving Hard Drive Performance

Max Dunn — Wed, 13 Sep 2006 20:37:59 -0700

My computer is pretty lame. It is a lightweight Averatec 3280 laptop running Windows XP, and I bought it at Staples for under $900. This is half the price of a comparable Dell, so I willing to live with its problems. One of the problems is that whenever it hibernates, it comes back up with the wireless card off, so I need to turn it on then search for a wireless connection. Another problem is that the hard drive keeps dropping into PIO mode which makes everything really slow. So far, I have been deleting the IDE controller and rebooting twice to fix it, but this is a lengthy process.

Thanks to James Barton, here are some instructions for getting around this problem. I am going to try this on my computer and see how it works:

Open up the Registry Editor.
Navigate to the following key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet \Control\Class\{4D36E96A-E325-11CE-BFC1-08002BE10318}.
There are several sub-keys under this one, such as 0000, 0001, etc. You are interested in two of them that say Primary IDE Channel and Secondary IDE Channel.
Make the following changes to both of those keys:
1. Delete any attributes named MasterIdDataCheckSum or SlaveIdDataCheckSum. This resets the tracking for errors that Windows uses to determine when the transfer mode should be lowered.
2. Add an attribute with the name ResetErrorCountersOnSuccess and a DWORD value of 1. This tells Windows that it should lower the transfer mode when there are six consecutive errors instead of six cumulative errors.
3. If they exist, set the following keys to a hexadecimal value of ffffffff (eight F’s). This will change the transfer modes to UltraDMA-6:
  - MasterDeviceTimingMode
  - MasterDeviceTimingModeAllowed
  - SlaveDeviceTimingMode
  - SlaveDeviceTimingModeAllowed
  - UserMasterDeviceTimingModeAllowed
  - UserSlaveDeviceTimingModeAllowed
Reboot your computer and check the devices to see if they are set to UltraDMA Mode 6.

Test Post From Windows Live Writer

Max Dunn — Thu, 24 Aug 2006 15:24:00 -0700

This is a test of the Windows Live Writer. It works pretty well!

On-Demand Document Management

Max Dunn — Fri, 18 Aug 2006 11:05:00 -0700

Here are some of my thoughts regarding on-demand document management.

How To Really Mess Up a Hard Drive (Then Fix It)

Max Dunn — Fri, 07 Jul 2006 11:05:00 -0700

Yesterday, I almost lost everything on my hard disk – very scary. I was playing around with Ubuntu 6.06 and it ran well from the CD, so I let it install itself to the hard disk. I knew that it was going to partition the disk and add a boot loader, but I figured that the worse thing that could happen was that I would have a boot menu and be able to select to boot from Windows each time. Wrong. I almost lost all my data, but in the end, was able to restore the original partition information. Here is my tale and how I recovered.

Getting Started With Flickr on Rails

Max Dunn — Thu, 06 Jul 2006 10:47:00 -0700

I have been working on a Flickr mashup on my wiki so that I can easily insert links to my Flickr pictures. There are several Flickr libraries for Rails, but the documentation is not very good. So here are some instructions for getting a Flickr library working with Rails.

(Note: On August 28, 2006, I expanded and updated these instructions at: http://www.maxdunn.com/RoR+and+Flickr)

Eclipse and Subversion

Max Dunn — Wed, 05 Jul 2006 13:41:00 -0700

It can be a little confusing to set up Eclipse to work with your Subversion repository. Once you have downloaded and installed Subclipse, follow these instructions to connect to your SVN repository:

Google Map Mashup in Rails

Max Dunn — Fri, 30 Jun 2006 10:03:00 -0700

How would you like to put an interactive Google map in your Ruby on Rails page with single or multiple markers? Here is how to do this.

Eclipse: Closing Projects

Max Dunn — Tue, 27 Jun 2006 19:25:00 -0700

Eclipse is known to be a memory hog and slow at starting up and shutting down. Especially if yo have a lot of projects with a lot of files, it can be painfully slow to start. A friend of mine took a programming class where they had exams using Eclipse. All the students made sure they started Eclipse well before class, otherwise they joked that it would take half of the exam time just for Eclipse to start!

Today, I discovered something that helps speed up Eclipse. If you right-click on the projects that you aren’t actively working on and “close” them, Eclipse won’t try to load them into the workspace. This helps not only with the startup and shutdown times, but also when it refreshes the workspace.

Alfresco architecture

Max Dunn — Sat, 15 Apr 2006 10:58:00 -0700

Alfresco is a new but promising open source document management system. It is Java based and supports many of the latest standards including the JSR-170 file access API, the JSF tag based interface, the Spring framework, JSR-168 portlets and WebDAV file transfers.

While Alfresco is still young and lacks some functionality, it is well architected and shows a lot of promise. Here is some information about its architecture.

HDTV Setup

Max Dunn — Mon, 09 Jan 2006 22:50:00 -0800

HDTV (High Definition TV) is so cool, once you see it, you will never want to go back to regular TV. It is like the difference between listening to a weak radio station in the car that has a lot of static, and listening to a crystal-clear CD.

Watching Lost in HDTV, you can see the pores on the actors faces (luckily, there are some nice looking faces you don’t mind seeing close up) and the scenes of the ocean and jungle makes you feel like you are there in Hawaii with them (whoops, I mean some uncharted island) feeling the ocean breezes and tropical humidity.

We don’t watch any non-HD shows—it just isn’t the same. Conversely, there are some shows we watch just because they look gorgeous in HDTV, that we wouldn’t normally watch (like Las Vegas).

The kicker is that we don’t need to pay for cable to get HDTV; we just put on antenna on the roof and get all the local stations for free!

Now setting up an HDTV system is a little complicated, so here is some information on how to do it.

Linux Setup

Max Dunn — Sat, 10 Dec 2005 16:09:00 -0800

When we moved in September, I knew that I wouldn’t be able to keep the Internet connection to my home server going continuously, so I transfered all the email and web accounts I was hosting to a 1and1 account. I also wanted to update to the latest Mandrake Linux, to get rid of the hodgepodge of application versions I was currently running. So over the last few months, I wiped out my server, updated to Mandrake 2005 and proceeded to reconfigure it, especially as an router with web filtering software.

There were a few tricky things that took a while to figure out, so I wanted to document them in case I have to go through this exercise again.

But rather than clutter up this blog with all the excruciating details, I am going to add them as extended content to this entry. I doubt these notes will be of much use to anyone else, so consider this just an entry of me talking to myself. ;-)

Formatting Text in Typo

Max Dunn — Thu, 01 Dec 2005 09:56:00 -0800

This Typo blog is setup to use Markdown, and here are some basic formatting rules.

Input:

Header 2 ——- ##Another Header 2 emphasis strong * List 1 * List 2 – Another List 1 – Another List 2 1. Ordered List 1 2. Ordered List 2

Output

Header 2 ——-

##Another Header 2

emphasis

strong

List 1
List 2

- Another List 1 - Another List 2

1. Ordered List 1 2. Ordered List 2