machdb development blog

0.5.2 Release, SVN & Redmine

2009-04-17T01:48:00.001-07:00

I'm happy to say that for the last 2 months I've been spending some of my spare time working on MachDB again. Before I started coding again, I wanted to get an SCM selected and set up, and an issue tracker of some type. Initially I had planned on going with Subversion and Bugzilla, but after some convincing from Andreas Unterkircher, I decided to go with Redmine instead of Bugzilla. Took me some time to get it all set up, but we now have both source control and Issue Tracking/Project Management. It's really nice, I may have to make a case for this at work too. It has RSS feeds for everything, so you should be able to use it to keep up to date on all aspects of the project.

http://tools.unmx.com/redmine/projects/show/machdb

Please use this going forward for opening bug reports, feature requests, support questions, submitting patches, etc.

I'm also happy to announce that I have been fixing bugs and have released MachDB 0.5.2 tonight. I fixed the long standing package insertion bug, which I know a particular co-worker of mine will be very happy about. Go get the release.

You can also see my project roadmap here, release dates, features and bugs listed for each

Feel free to contact me, I like to know who's using MachDB and what improvements you'd like to see. And as always, I appreciate feedback of any kind.

Lot of interest lately

2009-01-23T05:28:00.000-08:00

I've been seeing a lot of interest in the project lately. I hear there was discussion of MachDB on a Ruby mailing list. That's great! I have had a lot of work and side projects over the last year that have taken me away from development on MachDB, but I want to assure the user community that it's not forgotten. I've had a number of good bug fixes submitted by users, and a few awesome contribs by a few people (thanks Andreas!). SVN and RedMine are getting set up now and all these patches will be rolled in.

Short term road map:

SVN
RedMine
Mailing List
Bug Fixes
A few new Self-Checks for the host scripts
Some of Andreas's Changes

Mid-Term:

Tags
Custom XML

Long Term:

Web Services API
CLI Client

Any questions, just hit me up. Thanks everyone for the support and the nice emails.

LISA 2008 San Diego

2008-11-12T11:07:00.000-08:00

It's been a while since I last posted to this dev blog..not a lot of activity in the last year. As LISA 08 has approached, I've had a renewed inspiration in MachDB and am excited to be working on it again. I'm at the LISA conference Wed, Thurs & Friday if anyone would like to meet up and discuss the future of the MachDB, features you'd like to see or gripes/complaints. Hit me up on the blog and we'll find a way to meet up!

Web 2.0 Expo in SF

2008-04-22T15:28:00.000-07:00

I'll be at the Web 2.0 Expo in San Fran this week, Wed-Friday. Hit me up if you're there and want to talk MachDB or just get a beer!

I've been busy philosophizing with others about the future direction of the MachDB. I have some code to write and some cool new features coming soon. I'll post more after the conference is over.

Freshmeat and Sourceforge

2007-12-26T03:23:00.000-08:00

We now have some more project features, courtesy of the nice people at freshmeat.net and SourceForge.

http://freshmeat.net/projects/machdb/

and

https://sourceforge.net/projects/machdb/

I'm not using a lot of the features at SourceForge, but the bug and feature tracking systems will come in handy...if I can get people to use it. As for Freshmeat, I'll use that to get info out to the community.

Documentation: Data Flow Diagram

2007-12-22T04:11:00.000-08:00

In the interest of education, I have been working on docs. I started off with a data flow diagram, available in PDF and JPG form.

machdb-0.5.1 released

2007-12-19T23:19:00.000-08:00

I put out a new version tonight of the code and a slightly updated XML spec. The following bugs and issues have been addressed:

Filesystem and disks have had the limit raised from 2TB to 16 zettabytes.
OS Name limit now 100 characters
Added Base Architecuture to host node
Added Unified Cache to CPU node
Split up 'version' and 'release' into 2 different fields for packages. 'release' is optional input
Various UI bug fixes
Renamed files, added contrib directory, in an attempt to standardize

To update your DB scheme, please run the following:

mysql -u mysqluser -p mysqldatabse < update_0.5.0-0.5.1.sql

To update the data in your tables and your schema at the same time:

php update_0.5.0-0.5.1.php

This will update your packages table to include release information. If you don't want this, comment it out. If you do, you'll want to check your gathering scripts to make sure they also sent the release information properly.

I also added some contributed gathering script files. We now have them for PPC Linux, Sparc Solaris and Debian i386. Thanks to Chris and Kyrre for those contributions!

Bugs, Specs and new Sites

2007-12-14T01:23:00.000-08:00

Moved to a new website, machdb.org Please update your bookmarks. I hope the URL change will make it into the Feb issue of LOGIN. Either way, I'll set up redirects at the old site to get people here. I also moved the blog, to be more central. When I get some time I'll make a new wiki on this domain as well.

Bugs!

Fixed an XML parsing bug in collect.php, thanks to the quick eyes of Josh, Steve and Stephen.

Found a big bug in my spec thanks to Chris. From my response:

"So I found out why your disk data showing a max of 2TB. See...this is where seeing it run in other environments teaches me the right way to do things. ;) Apparently I had some bad math when I calculated what MySQL's int(11) datatype could do. So I guess I've either got to modify the MySQL tables to be BigInt or change the spec to start with MB. With BigInt and KB, the limit is 9,223,372,036,854,775,807 KB, or 8,192 exabytes. Yep, I said exabytes. But that makes the DB bigger. With int(11) and MB, the limit is 2,048 terabytes. Thoughts? I'll noodle on this tomorrow and run it by a DBA buddy and get his feedback as well. Until then, sit tight. I will have this resolved in the next day or three."

I updated the XML spec just now to describe the limitation, so as not to confuse others.

Chris has also sent me a patch to include a unified cache in the CPU information, which will go into the next release next week. I'd like to get this out before Xmas, since I won't be around for a bit afterwards.

These bug reports are great everybody, keep em coming. :)

XML Spec Published & Wiki

2007-12-10T23:33:00.000-08:00

After the release last night, I sent the code to a fellow sysadmin from LISA who had e-mailed me with interest in the MachDB. He's already got it running, and with not too much effort. My first user, cool! I'm very excited to see it working somewhere else other than my 2 environments. To that end, I worked diligently tonight to document the XML spec that describes what info you can send to the MachDB. You can find the doc linked off the main project website, or you can click here.

Comments on this are welcome, so let me know what you think.

Also, I set up a temporary wiki page at http://www.mybrainhurts.com/wiki/MachDB. You can go there for information from other people that have installed it. I will be keeping a FAQ and install instructions there. Once the permanent wiki is up, all docs will go there.

And if anybody knows of an easy way to syntax highlight code on a web page, you must tell me. ;)

Code is out!

2007-12-09T23:25:00.000-08:00

Yes, yes, 2 weeks longer than I expected...thanks for your patience. I had to fix a few last minute bugs and clean it up for general consumption. My instructions are not complete, and I have yet to define the XML spec (my next task), but the code is out! Go download version 0.50 at the project site: http://www.machdb.org

OK, so what is next. Here is my near term task list:

Document XML Structures
Improved Search
Machine Readable Interface
Tags, Comments, Logging
Administrative Tools (audits, etc)
Host vs Host comparison. Mmmmm, Tasty.

I am very much interested in ideas, feedback (good and bad) and bug reports. How can it be improved? And if you use it and like it, let me know too!

Mmm, Turkey

2007-11-29T03:05:00.000-08:00

Well, Thanksgiving is over and I've had a bit of time to apply to MachDB. I spent a few hours tonight digging back into the code. It's funny how fast you forget what you've worked on. Fixed a few bugs that I found, and worked on syncing everything to the latest greatest.

I've also got to figure out how to use the UUID instead of the MAC for hosts that have that. And provide a way for it to change. Should it be automagic? I don't know, I almost feel that it shouldn't.

I'll upgrade VMIX to the latest version of the dev code and at the same time I'll put out the release. w00t! Almost there people, bear with me! :)

Home for the Holidays

2007-11-21T02:15:00.000-08:00

Well, I didn't get time this week to clean up the code. If I have any downtime/alone time while I'm visiting with family over Thanksgiving, I will see what I can get out. Otherwise, look forward to something in late Nov/early Dec.

I must now finish packing, attempting to travel on the worst (travel) day of the year means being extra organized. T minus 7 hours till my plane takes off. And I still need sleep.

Have a good holiday!

Website up

2007-11-19T03:19:00.000-08:00

I spent tonight putting up a up a website. Thank goodness for OSWD. Now I can communicate with the people that want to know about the MachDB. Yay! I also put up some basic info and a list of current features. Still need to put up the planned features, but need to decide on the rank and order first. Hopefully, if this work week goes well, I can have a night to work on the code cleanup and get something out before Thanksgiving.

LISA 2007 - Some new ideas

2007-11-18T17:19:00.000-08:00

At LISA 2007 this year, I gave a WIP (Work In Progress) talk on the MachDB. I got some positive feedback on it, and have been inspired by the things people have said. I was told that the chatter on the IRC back channel at LISA was good, I'd be interested to see that if anyone has a log. Otherwise, I have taken a few notes on things I'd like to do:

Statistics - Show totals of neat stuff like RAM, CPU, BogoMips, Disk Space, Uptime, IP counts, etc. Use the cool graphing tools I have to show graphical representations of these things. Possibly add RRD data storage on that data, or at least a stats table in the DB.
Auditing - Find things in the DB that shouldn't exist, or are wrong.
Admin Tool - Ability to change a host's unique identifier (eth0 MAC or UUID). Also, need to be able to delete entries and merge entries in the case of a host with bad data getting in the system.
Glue - A completely customizable page that lets MachDB be the 'glue' between every other web app in your sysadmin team. Stuff like Nagios, MRTG, Cricket, Cacti, Smokeping, etc integration. This is mostly going to just be a page that has example glue for these projects, and is intended to be completely customizable.
Fettle Integration - Andrew Hume did a pretty cool presentation on 'Fettle', a tool for visualizing rack space layout in 3D/WRML. I noticed that many of the things he needed to generate his files are things I have in my DB. This means MachDB could auto-generate these config files. Mmmmmmm.

This is not a list of features that are actually planned. I'll post that later. ;)

new blog!

2007-06-14T02:59:00.001-07:00

I decided to start blogging my development on MachDB. Seems the appropriate thing to do.