Saturday, November 8, 2008

Proftpd & MySQL

Recently I re-worked the way that our FTP servers run at work. We use Proftpd in conjunction with MySQL for authentication. The setup required some reading from the Proftpd documentation, so I figured I could help someone out by showing our configuration found in the "proftpd.conf" file and what our database looks like.

First, the MySQL database has three tables: hosts, login_log, and users.


CREATE TABLE `users` (
`id` int(11) NOT NULL auto_increment,
`host_id` varchar(50) NOT NULL default '0',
`username` varchar(100) NOT NULL default '',
`passwd` varchar(100) NOT NULL COMMENT 'use PASSWORD() to encrypt',
`uid` int(5) NOT NULL default '5000',
`gid` int(5) NOT NULL default '5000',
`ftpdir` varchar(255) NOT NULL default '',
`ts_created` timestamp NOT NULL default '0000-00-00 00:00:00',
`ts_modified` timestamp NOT NULL default CURRENT_TIMESTAMP,
`deleted` smallint(1) NOT NULL default '0',
PRIMARY KEY (`id`),
UNIQUE KEY `username` (`username`)
) DEFAULT CHARSET=utf8;

CREATE TABLE `hosts` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(50) NOT NULL default '',
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) DEFAULT CHARSET=utf8;

CREATE TABLE `login_log` (
`ip_address` varchar(15) NOT NULL default '',
`dns_name` varchar(150) default NULL,
`timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP,
`username` varchar(100) NOT NULL default ''
) DEFAULT CHARSET=utf8;


The "hosts" table is created with the intent that multiple hosts use this database for FTP authentication. Insert the FQDN of all hosts into the "name" field.

The "users" table holds all the pertinent user information. The "password" field is encrypted using the MySQL "PASSWORD" function. "ftpdir" holds the directory to chroot the user into. "uid" and "gid" are the respective user and group id to assign to files that are created on/uploaded to the server. "host_id" is used to specify which server this user can log in to. It ties back to the "hosts" table.

The "login_log" table is used for logging whenever a user logs into the server. "ip_address" stores the IP address of the person who logged in. "dns_name" is the contents of a reverse DNS lookup on the IP address. "username" is the name of the user that logged in.

Now for the actual "proftpd.conf" configuration.


ServerName "FTP Server"
ServerType standalone
DefaultServer on
Port 21
Umask 007
MaxInstances 30
User nobody
Group nogroup

This group of directives sets up some of the basics for the FTP server. Proftp suports Virtual Hosts in manner like Apache.
"DefaultServer" tells proftpd to use all the settings here as the default server when a connection comes in.
"ServerType" tells the Proftp daemon to listen for connections instead of using inetd.
"Port" sets the port to use.
"Umask" sets the default permissions.
"MaxInstances" sets the maximum simultaneous connections.
"User" and "Group" set the user and group to run the server as.


SQLEngine on
SQLBackend mysql
SQLAuthTypes Backend
SQLConnectInfo mysql_user@mysql_host db_name mypassword
SQLAuthenticate users
SQLUserInfo custom:/select_user
SQLNamedQuery select_user SELECT "username, passwd, uid, gid, ftpdir, '/bin/bash' FROM users WHERE deleted=0 AND username='%U' AND host_id=host_id_for_this_host"


"SQLEngine" tells the server to turn on the abililty to use a SQL backend.
"SQLBackend" should be set to "mysql" to use MySQL as the authentication backend.
"SQLAuthTypes" tells the server what encryption is used for the password in the backend. The value "Backend" tells it to use the "PASSWORD" function for encryption.
"SQLConnectInfo" gives the connection info for the MySQL database.
"SQLAuthenticate" specifies the table to use for authentication.
"SQLNamedQuery" is used to specify a query to be used in some other location. For instance, this one is called "select_user". Looking at the "SQLUserInfo" line it can be seen that we use a custom query to authenticate users. The "select_user" query is used whenever a user attempts to log in. The columns MUST be returned in a very specific order. See the "SQLUserInfo" directive in the Proftp documentation for more information.


SQLDefaultUID 65534
SQLDefaultGID 65534
SQLMinUserUID 20
SQLMinUserGID 20
SQLLog PASS insert_login
SQLNamedQuery insert_login INSERT "'%a', '%h', NOW(), '%U'" login_log


"SQLDefaultUID" and "SQLDefaultGID" are the user id and group id to use if none are specified. These values will probably never be used.
"SQLMinUserUID" and "SQLMinUserGID" are the minimum values allowed in the authentication table for a user. If these minimums aren't met the user will not be able to log in.
"SQLLog" tells Proftp a query to run when the given FTP command is given. In this case, it logs anything that successfully authenticates to the FTP server.
"SQLNamedQuery" again specifies a query to run when called. See "SQLNamedQuery" in the Proftp documentation for more details. To know what the different variables mean in the query, see "LogFormat" in the Proftp documentation.


SystemLog /var/log/proftpd/proftpd.log
DefaultRoot ~
AllowOverwrite on
AllowRetrieveRestart on
AllowStoreRestart on
UseReverseDNS on
IdentLookups off
ListOptions "-a"
TimeoutIdle 3600


"SystemLog" tells where to log all of the Proftpd messages.
"DefaultRoot" jails the user into their ftp directory when set to "~".
"AllowOverwrite" allows files to be overwritten.
"AllowRetrieveRestart" allows a file to restart a download from where it last finished.
"AllowStoreRestart" allows a file to restart an upload from where it last finished.
"UseReverseDNS" tells Proftp to do reverse DNS lookups.
"IdentLookups" tells Proftp whether or not to do Ident lookups.
"ListOptions" tells Proftp what options to pass to an "ls" command by default.
"TimeoutIdle" is the timeout before automatically disconnecting an idle user.



Quite a write-up, but I hope it helps someone.

Sunday, October 26, 2008

SVN & MySQL

Just a week ago, my company migrated to SVN from CVS. I won't go into a discussion of why or what the differences are, but I thought it would be worth writing about our configuration of SVN. Before getting started, I should point you to the free, excellent SVN book found at http://svnbook.red-bean.com/. If you wanted to know about any of the features or configuration options available in Subversion, that's a great place to look.

There were three things that I wanted to accomplish with SVN:

1. Authentication using a MySQL database
2. The ability to manipulate access control at any level in the repository using the MySQL authentication (ie. Only allow access for a user to /client/trunk/project, not to the whole trunk)
3. The connection had to be HTTPS. I don't want our source code going over the internet in plain text.

First, let's cover the basics. Since I'm coming at this from the point of view of a Gentoo installation, I'll cover what's needed for that. The "dev-libs/apr-util" package must be compiled with the "mysql" USE flag enabled. As for apache, I just compile it with all modules enabled. This is easy to accomplish by setting APACHE2_MODULES="*" in /etc/make.conf. As for the MPM to use for Apache, we've been using the "prefork" one. I can't really give an argument for any given MPM since I don't know enough about performance of them to argue a point.

The second step is fairly obvious. You have to compile Apache. It should drag in the apr-util package as a dependency, but if it doesn't for some reason, make sure you compile that package as well. A simple "emerge apache" will do the trick.

The third and final step is to set up the configuration. I'll step through the configuration for our server to give a flavor of what it could look like. You can throw this stuff into a Virtual Host if that is what you want. It's what I did.


#I realize that the document root points to somewhere other than the SVN repositories. This is intentional. It's the only way I could get Subversion to work the way I wanted it to.
DocumentRoot /var/www/localhost/htdocs/

#Enable SSL. It's as simple as turning the engine on and providing the paths to the cert and the key
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain.key

#This is the real meat and potatoes of the whole thing.

#This enables the mod_dbd MySQL driver
DBDriver mysql

#This causes mod_dbd to maintain the MySQL connection after every query instead of reconnecting for every MySQL call.
DBDPersist On

#These are the basics for a MySQL connection. Notice that the whole line is surrounded in quotes. See http://httpd.apache.org/docs/2.2/mod/mod_dbd.html for other parameters that can go on this line.
DBDParams "host=mydatabase.server.com user=myuser pass=mypassword dbname=mydatabasename"


# The location "/" causes SVN to interpret anything that follows the "/" as a repository path
<location />
DAV svn #Treat the location as a subversion repository
SVNParentPath /var/svn/ #This treats any directory under "/var/svn/" as the root of a repository
Allow from all #Allow access from anywhere
AuthName "Login" #The title of the authentication window
AuthType Basic #Not quite sure what this does, but it's necessary
AuthBasicAuthoritative Off #Tells Apache to try other authentication methods if this one doesn't work
AuthBasicProvider dbd #Tells Apache to try mod_dbd authentication first
</location>

#A regex that is used to determine the query to use for my current location
<locationmatch ^/client/trunk|tags|branches/project>
# This location requires an authenticated user. AuthDBDUserPWQuery determines how apache decides if the user is valid. The password in MySQL must be stored using the ENCRYPT() function. The supplied password must mach the stored ENCRYPT() password.

Require valid-user
AuthDBDUserPWQuery "SELECT password FROM users WHERE user=%s"
</locationmatch>


That's about it! Not the neatest write-up, but I hope this helps someone. I know I had a heck of a time trying to get it all working correctly. You can add as many locations as you want for different clients and/or subdirectories.

Edit: Something I found out later is that apr-util v1.3.x compiled with MySQL support and PHP compiled with MySQL support don't mix. It will cause Apache to segfault and puke all over the place. Use apr-util v1.2.x to avoid this problem (see https://bugs.gentoo.org/show_bug.cgi?id=240264 for details on the issue).

Thursday, October 9, 2008

Fun With Apache Fix

About a month ago I wrote about a weird problem that I was having with Apache in regards to mod_dbd and the MySQL driver. I never did write about what was causing the problem. After much troubleshooting, I discovered that Apache did not like the mod_dbd driver to be used with PHP if PHP had been compiled with MySQL support. I never could figure out why it was a problem, only that it was the cause of the segmentation fault.

Luckily, my company just recently started using VMware ESX server for virtualization, so I simply made another virtual machine to use for Apache with mod_dbd. SVN uses mod_dbd as a backend for authentication and it works beautifully. Come to think of it, I probably should write up an entry on how to configure SVN to work the way I did. It took me long enough to figure it out, that's for sure.

Sunday, September 28, 2008

101 Things in 1001 Days

My brother-in-law came up with the idea of make a list of 101 things to do in 1001 days. I didn't realize this was so popular, but a quick search on Google comes up with tons of lists. In any case, Aubrie, Matt, and I are all going to be doing this starting October 3, 2008. Prior to popular opinion, Aubrie did not give any of the ideas on here. Here's my list (or at least what I've thought up so far):

1. Get Lasik done on my eyes
2. Play through any video games purchased within the last two years that I haven't finished yet
  • Pirates
  • Portal
  • Far Cry
  • Zelda: Twilight Princess
  • Rayman Ravin' Rabbids 2
  • Unreal Tournament 3
  • Lost Planet: Extreme Conditions
3. Use Linux as my primary OS for 3 months. (in progress 10/20/2008)
4. Build a MythTV box for home.
5. Tear the tree out of our backyard (in progress)
6. Dig out the garden in the backyard
7. Fix the leaking valve in the sprinkler system
8. Clean the BBQ Grill
9. Dig out the flower beds in the front and back yard
10. Build shelves in our garage
11. Repaint the master bedroom
12. Repaint the computer room
13. Put cabinets in the laundry room
14. Put more cabinets in the kitchen
15. Put our TV on a wall mount
16. Get a surround sound system for the Family Room
17. Put new carpet in the bedrooms
18. Buy a new lawn mower
19. Cook dinner for Aubrie 5 nights in one week
20. Be in bed ready to sleep by 10pm every night for 2 weeks
21. Fix the tree in the front yard (done - 10/24/2008)
22. Put shelves up in the Family Room
23. Build shelves in the game closet
24. Figure out some way to finish the crawl space and do it
25. Hang stuff up in the computer room
26. Get Aubrie a new laptop
27. Build a web server for home
28. Vote


More forthcoming.....

Friday, September 26, 2008

Fun with Apache

Over the last week I've been busy getting a new MySQL server and Apache server built on our new VMware ESX server at work. Everything has gone smoothly with the exception of one thing. For the life of me, I cannot get Apache to cooperate. For our new SVN repository, I'm using the mod_dbd module for MySQL authentication, but every time I enable it using "DBDriver mysql" in httpd.conf I randomly start getting errors like the following on start and stop of Apache:

/etc/init.d/apache2: line 43: 16808 Segmentation fault ${APACHE2} ${APACHE2_OPTS} -t > /dev/null 2>&1

The weird thing about it is that it occurs randomly. Sometimes I can stop and restart without any problems at all. The second I remove the DBDriver line, everything works great. What makes it even stranger is that I have a 32 Bit virtual machine on my laptop that uses the module without any problems at all. Our server is running 64 Bit Gentoo, but I have gone to the point of making everything identical (with that one exception). Does anyone have any ideas what could cause this problem?

Saturday, September 13, 2008

Previously on Geek Speak

About 4 months ago I wrote an entry saying I would write up some decent Gentoo packages for a desktop system. While this won't be for a desktop system, I will bring up some good packages for a Gentoo server system. Some of them are obvious, but there are a few that took me a while to find.
  • logrotate - Rotates log files on the server. Comes with some pre-scripted files to rotate, but can also be customized for any log file.
  • metalog - System logger. Can be extended to dump different system events into separate log files.
  • screen - Emulates a terminal. Essentially, it can be used to start programs that need to maintain a session open. I use it all the time for compiling things in the background when I don't want to maintain an SSH connection to a server
  • genlop - Used to find information about currently emerging and already emerged packages. It can give an estimated time to finish an emerge and tell how long an emerge has been going.
  • gentoolkit - This is a must have for Gentoo. It provides various scripts for maintaining a Gentoo install. For instance, there is a program called "revdep-rebuild" which will verify that all emerged binaries link against the correct libraries on the system.
  • iptraf - This is a handy app that can be used for network monitoring.
  • tcpdump - This program can be used to monitor packets coming into a specified network interface.
  • logwatch - Can be used to email a summary of the logs on the system.
  • slocate - Indexes files on the filesystem to allow for quick searching
This is by no means an exhaustive list of helpful packages.

Monday, September 8, 2008

When CVS just won't cooperate...

Last week was pretty hectic for me. It all started Tuesday when our CVS server would freeze on a CVS add. The whole machine would lock up and the only thing that could be done was a hard reboot. Needless to say, this was a less than desirable method of development. It was really weird too. CVS commits and updates of files that already existed worked great, but the CVS add function would freeze the server almost every time. Seeing as how I couldn't find ANYTHING (I literally mean that) on my specific problem on google, I thought I would post my solution.

First I tried simply building a new Gentoo Linux server since that is my preferred Linux distro. I migrated all of the CVS data over to the new server along with all the permissions and ACLs that were assigned to the repositories so that it would be a simple migration process. Unfortunately, this didn't work. The new server exhibited the same behavior as the old one. This pointed me in the direction of three things possibly being wrong
1. The CVS repository was corrupt (I really hoped it wasn't that one)
2. The ACLs were somehow all screwed up
3. Some library or binary in Gentoo had a bug that was causing the problem

Since I was mostly interested in just getting things working again, I decided to go the route of 2 & 3. I spent part of a day building a Fedora Core 9 server and again copied the CVS repository to the new server, but this time I left out all the ACLs. It was somewhat of a pain, but I went through and recreated the ACL structure on the whole repository. The end result was a working stable CVS that I haven't had problems with since I migrated it to Fedora. I still don't know exactly what the problem was, but at least this worked for me. I hope it does for someone else as well.

An oldy, but definitely a goody

As someone that is a fan of first-person shooter games, I'm always on the look out for a good game of that genre. Anyone that's a fan of this genre has more than likely heard of Crysis. Now, the purpose of this post isn't a review of the game, but I want to point out three things about this game.

First of all, the visuals are amazing. If you don't believe me, look up some screenshots of the game in it's full detail glory. I never knew computer graphics could generate a sunrise that looked so amazing. It does require quite a beefy video card, but since it came out the prices on video cards have kept dropping. I use an Nvidia 8800 GTS 512MB card and it runs the game quite well at pretty high detail levels.


I found this picture on-line in a forum. It isn't mine.

Second, the story is very well done. I don't know of many first-person shooters that are known for great stories. About the only one that comes to mind is the Half-Life series (another excellent series). The story kept me wanting to play and find out what was going to happen next.

Now, the third and final thing about this game is why I wrote this post. Be warned that the language in this game is absolutely awful. I purchased the game, played through it, then returned it because the language was so bad. Luckily, I happened to know a person at school that discovered a way to edit the language content. So I ended up purchasing the game again and editing the audio. I have no idea if posting my edited language pack (I call it my clean language pack) would be a violation of copyright law, so I won't post it here. I spent a few hours going through all the mp2 files in the game and editing out the bad language in the single player campaign. If you want a copy, just post a comment and I'll see what I can do about getting it to you. I think everyone should have a chance to play this amazing game without listening to the constant barrage of F-words.

Sunday, September 7, 2008

3 nifty Vista features

Aubrie decided this week that we should go get some library cards. We got them yesterday and I must admit that I've got a lot of reading ahead of me. Along with some other books, I got a book called "Windows Vista: Beyond the Manual." At first, I thought it would be a pretty in depth book about the internals of Vista, but for the most part, I was wrong. Let's just say there were approximately 454 pages out of 459 that I just didn't care about. I knew most of the stuff already (I did a LOT of skimming and skipping), but there were three things that I thought were kind of cool.
  • Have you ever tried selecting 10+ files from a folder that are all over the place? You have to hold CTRL while selecting each file and if you accidently let go of that button you have to start all over. Apparently, Vista has an option in the "Folder Options" box called "Use Check Boxes to Select Items." This lets you simply select check boxes instead of holding CTRL and screwing it up. Not earth shattering by any means, but somewhat useful.
  • Second, one short coming of Windows in general has been the lack of symbolic links. Well, I guess Microsoft finally wised up and added the functionality. It can only be done from the command line for now, but I guess it has to start somewhere. Using the "mklink" command, links can be made to folders and directories.
  • Lastly, I'd never heard of this before, but it seems Vista has an automatic error recovery program that can be run on boot-up if the system is in an unbootable state. It's called WinRE. I haven't tried it out yet, but it looks like it could be fairly useful for those that aren't too knowledgeable about computers.
I hope this benefits somebody, because I hope I never have to read/skim/skip through that many pages of text just to find three useful things, but I guess learning isn't always easy.

Who wants some eye candy?

Lately I've been taking yet another dive into using Windows Vista. I've used it on and off since the release candidates were available in both the 32 and 64 bit versions. One of my biggest complaints has always been that it seemed like most of the software that I use all the time has incompatibilities with Vista. Also, Microsoft did something that many people have complained about with the 64 bit version. They enforce driver signing. In other words, any driver that has not been approved by Microsoft cannot be used. Also, Vista is a resource hog. Honestly, why should an operating system need 768MB to 1 GB of my memory just to load? With those things in mind, I decided to give Vista a try yet again and I must admit that it hasn't been as bad as the last few times.

First of all, most of my programs seem to work now. Either there are patches for them that give them Vista compatibility or the open-source equivalents give me the same functionality. I guess after an OS ages for a year and a half, things actually start to work (shouldn't that be the case earlier?).

Second, this time around I decided to give 64 bit a try again and was pleasantly surprised. Somebody decided to stick it to Microsoft (however minor it may be) and got around the stupid driver signing enforcement. ReadyDrive Plus installs itself into the MBR and automagically selects the Windows boot option to disable to the signing enforcement. Once I put that on there, all my drivers work well.

Third, I don't see Windows as quite as much of a resource hog now that my Laptop has three GB of RAM and my desktop has four. Besides, everyone has to admit that the GUI does look good even if the looks don't provide extra functionality.

Anyone who knows me knows that I've never been a proponent of Vista. I've preferred Windows XP for a long time now, but it seems like that might be changing. Scary, huh?

Thursday, May 22, 2008

Gentoo Setup

I was doing some searching the other day on packages for Gnome and other things in Gentoo for a decent desktop system. I couldn't really find anything so I think that I'll impart my limited knowledge on the matter with the hopes that it will be useful to someone. This will happen over the next several blog entries

Sunday, April 20, 2008

What's One More Pound?

If there's one thing that I've learned from my wife, it's that food was meant to be enjoyed. She loves to think of any excuse to have tons of good food. Just yesterday we had a "Freaks and Geeks" party in honor of the TV show by the same name. She came up with all kinds of food that might fit in for 1980. To name a few things there were chicken nuggets, cheese fondue, 2 kinds of chocolate fondue, pizza sauce fondue, veggies, pizza bites, potato wedges, tortellini, fruit, and more stuff that I can't think of. Needless to say, I feel like I gained a pound or two just from last night. Just think what I'll look like after 10 more years of this!

Sunday, March 23, 2008

Another One Bites the Dust....

I always thought it was weird that people would post their thoughts for the whole world to see, but here I am doing it (hence the title). This will more than likely end up as a dumping grounds for all the random stuff that I learn about random things (for instance, invisibility cloaks http://www.hanselman.com/blog/HanselminutesPodcast101DrMichioKakuOnThePhysicsOfTheImpossible.aspx).

Until I have time to put up a proper post (about computer things of course), you'll have to content yourself with this fun bit of geek speak.
http://www.flixxy.com/rockwell-automation-systems.htm