machdb devlopment blog

LISA 2008 San Diego

noreply@blogger.com (Nathan) — Wed, 12 Nov 2008 19:07:00 +0000

It's been a while since I last posted to this dev blog..not a lot of activity in the last year. As LISA 08 has approached, I've had a renewed inspiration in MachDB and am excited to be working on it again. I'm at the LISA conference Wed, Thurs & Friday if anyone would like to meet up and discuss the future of the MachDB, features you'd like to see or gripes/complaints. Hit me up on the blog and we'll find a way to meet up!

Web 2.0 Expo in SF

noreply@blogger.com (Nathan) — Tue, 22 Apr 2008 22:28:00 +0000

I'll be at the Web 2.0 Expo in San Fran this week, Wed-Friday. Hit me up if you're there and want to talk MachDB or just get a beer!

I've been busy philosophizing with others about the future direction of the MachDB. I have some code to write and some cool new features coming soon. I'll post more after the conference is over.

Freshmeat and Sourceforge

noreply@blogger.com (Nathan) — Wed, 26 Dec 2007 11:23:00 +0000

We now have some more project features, courtesy of the nice people at freshmeat.net and SourceForge.

http://freshmeat.net/projects/machdb/

and

https://sourceforge.net/projects/machdb/

I'm not using a lot of the features at SourceForge, but the bug and feature tracking systems will come in handy...if I can get people to use it. As for Freshmeat, I'll use that to get info out to the community.

Documentation: Data Flow Diagram

noreply@blogger.com (Nathan) — Sat, 22 Dec 2007 12:11:00 +0000

In the interest of education, I have been working on docs. I started off with a data flow diagram, available in PDF and JPG form.

machdb-0.5.1 released

noreply@blogger.com (Nathan) — Thu, 20 Dec 2007 07:19:00 +0000

I put out a new version tonight of the code and a slightly updated XML spec. The following bugs and issues have been addressed:

Filesystem and disks have had the limit raised from 2TB to 16 zettabytes.
OS Name limit now 100 characters
Added Base Architecuture to host node
Added Unified Cache to CPU node
Split up 'version' and 'release' into 2 different fields for packages. 'release' is optional input
Various UI bug fixes
Renamed files, added contrib directory, in an attempt to standardize

To update your DB scheme, please run the following:

mysql -u mysqluser -p mysqldatabse < update_0.5.0-0.5.1.sql

To update the data in your tables and your schema at the same time:

php update_0.5.0-0.5.1.php

This will update your packages table to include release information. If you don't want this, comment it out. If you do, you'll want to check your gathering scripts to make sure they also sent the release information properly.

I also added some contributed gathering script files. We now have them for PPC Linux, Sparc Solaris and Debian i386. Thanks to Chris and Kyrre for those contributions!

Bugs, Specs and new Sites

noreply@blogger.com (Nathan) — Fri, 14 Dec 2007 09:23:00 +0000

Moved to a new website, machdb.org Please update your bookmarks. I hope the URL change will make it into the Feb issue of LOGIN. Either way, I'll set up redirects at the old site to get people here. I also moved the blog, to be more central. When I get some time I'll make a new wiki on this domain as well.

Bugs!

Fixed an XML parsing bug in collect.php, thanks to the quick eyes of Josh, Steve and Stephen.

Found a big bug in my spec thanks to Chris. From my response:

"So I found out why your disk data showing a max of 2TB. See...this is where seeing it run in other environments teaches me the right way to do things. ;) Apparently I had some bad math when I calculated what MySQL's int(11) datatype could do. So I guess I've either got to modify the MySQL tables to be BigInt or change the spec to start with MB. With BigInt and KB, the limit is 9,223,372,036,854,775,807 KB, or 8,192 exabytes. Yep, I said exabytes. But that makes the DB bigger. With int(11) and MB, the limit is 2,048 terabytes. Thoughts? I'll noodle on this tomorrow and run it by a DBA buddy and get his feedback as well. Until then, sit tight. I will have this resolved in the next day or three."

I updated the XML spec just now to describe the limitation, so as not to confuse others.

Chris has also sent me a patch to include a unified cache in the CPU information, which will go into the next release next week. I'd like to get this out before Xmas, since I won't be around for a bit afterwards.

These bug reports are great everybody, keep em coming. :)

XML Spec Published & Wiki

noreply@blogger.com (Nathan) — Tue, 11 Dec 2007 07:33:00 +0000

After the release last night, I sent the code to a fellow sysadmin from LISA who had e-mailed me with interest in the MachDB. He's already got it running, and with not too much effort. My first user, cool! I'm very excited to see it working somewhere else other than my 2 environments. To that end, I worked diligently tonight to document the XML spec that describes what info you can send to the MachDB. You can find the doc linked off the main project website, or you can click here.

Comments on this are welcome, so let me know what you think.

Also, I set up a temporary wiki page at http://www.mybrainhurts.com/wiki/MachDB. You can go there for information from other people that have installed it. I will be keeping a FAQ and install instructions there. Once the permanent wiki is up, all docs will go there.

And if anybody knows of an easy way to syntax highlight code on a web page, you must tell me. ;)

Code is out!

noreply@blogger.com (Nathan) — Mon, 10 Dec 2007 07:25:00 +0000

Yes, yes, 2 weeks longer than I expected...thanks for your patience. I had to fix a few last minute bugs and clean it up for general consumption. My instructions are not complete, and I have yet to define the XML spec (my next task), but the code is out! Go download version 0.50 at the project site: http://www.machdb.org

OK, so what is next. Here is my near term task list:

Document XML Structures
Improved Search
Machine Readable Interface
Tags, Comments, Logging
Administrative Tools (audits, etc)
Host vs Host comparison. Mmmmm, Tasty.

I am very much interested in ideas, feedback (good and bad) and bug reports. How can it be improved? And if you use it and like it, let me know too!

Mmm, Turkey

noreply@blogger.com (Nathan) — Thu, 29 Nov 2007 11:05:00 +0000

Well, Thanksgiving is over and I've had a bit of time to apply to MachDB. I spent a few hours tonight digging back into the code. It's funny how fast you forget what you've worked on. Fixed a few bugs that I found, and worked on syncing everything to the latest greatest.

I've also got to figure out how to use the UUID instead of the MAC for hosts that have that. And provide a way for it to change. Should it be automagic? I don't know, I almost feel that it shouldn't.

I'll upgrade VMIX to the latest version of the dev code and at the same time I'll put out the release. w00t! Almost there people, bear with me! :)

Home for the Holidays

noreply@blogger.com (Nathan) — Wed, 21 Nov 2007 10:15:00 +0000

Well, I didn't get time this week to clean up the code. If I have any downtime/alone time while I'm visiting with family over Thanksgiving, I will see what I can get out. Otherwise, look forward to something in late Nov/early Dec.

I must now finish packing, attempting to travel on the worst (travel) day of the year means being extra organized. T minus 7 hours till my plane takes off. And I still need sleep.

Have a good holiday!

Website up

noreply@blogger.com (Nathan) — Mon, 19 Nov 2007 11:19:00 +0000

I spent tonight putting up a up a website. Thank goodness for OSWD. Now I can communicate with the people that want to know about the MachDB. Yay! I also put up some basic info and a list of current features. Still need to put up the planned features, but need to decide on the rank and order first. Hopefully, if this work week goes well, I can have a night to work on the code cleanup and get something out before Thanksgiving.

LISA 2007 - Some new ideas

noreply@blogger.com (Nathan) — Mon, 19 Nov 2007 01:19:00 +0000

At LISA 2007 this year, I gave a WIP (Work In Progress) talk on the MachDB. I got some positive feedback on it, and have been inspired by the things people have said. I was told that the chatter on the IRC back channel at LISA was good, I'd be interested to see that if anyone has a log. Otherwise, I have taken a few notes on things I'd like to do:

Statistics - Show totals of neat stuff like RAM, CPU, BogoMips, Disk Space, Uptime, IP counts, etc. Use the cool graphing tools I have to show graphical representations of these things. Possibly add RRD data storage on that data, or at least a stats table in the DB.
Auditing - Find things in the DB that shouldn't exist, or are wrong.
Admin Tool - Ability to change a host's unique identifier (eth0 MAC or UUID). Also, need to be able to delete entries and merge entries in the case of a host with bad data getting in the system.
Glue - A completely customizable page that lets MachDB be the 'glue' between every other web app in your sysadmin team. Stuff like Nagios, MRTG, Cricket, Cacti, Smokeping, etc integration. This is mostly going to just be a page that has example glue for these projects, and is intended to be completely customizable.
Fettle Integration - Andrew Hume did a pretty cool presentation on 'Fettle', a tool for visualizing rack space layout in 3D/WRML. I noticed that many of the things he needed to generate his files are things I have in my DB. This means MachDB could auto-generate these config files. Mmmmmmm.

This is not a list of features that are actually planned. I'll post that later. ;)

new blog!

noreply@blogger.com (Nathan) — Thu, 14 Jun 2007 09:59:00 +0000

I decided to start blogging my development on MachDB. Seems the appropriate thing to do.