Craig's Stuff tag:www.enc.com.au,2010-01-01:/sees-blog//1 2010-07-12T12:58:58Z Just a bunch of posts that do not to seem to fit anywhere else on the internet. Movable Type 4.34-en Manually calculating process times tag:www.enc.com.au,2010:/sees-blog//1.29 2010-07-12T12:06:18Z 2010-07-12T12:58:58Z Most of the time you can trust the times you see in programs such as ps and top within Linux, but there are those other times where you just want to check what is going on.First up, if you're using... Craig Linux, but there are those other times where you just want to check what is going on.

First up, if you're using a FreeBSD system and the procps Linux tools, you're out of luck because a lot of the procfs is broken.  You're almost guaranteed to get wierd results.

Jiffies and Hertz
The first thing to find is your jiffies.  This is the number of clock ticks per second.  There are two main ways of doing it.

The first, and more difficult way is to add up the cpu numbers in /proc/stat, divide it by the first number in /proc/uptime and divide it by the number of cpus.

My cpu line in /proc/stat looks like:
cpu  1267430 145826 552700 137029699 905086 8295 17885 0 0

and my /proc/uptime is:
695317.99 1370303.58

I've got 2 cpus on this system so the calculation is:

(1267430  + 145826 + 552700 + 137029699 + 905086 + 8295  + 17885 + 0 + 0 )  / 695317.99 / 2
which equals 100.

The second way is to use getconf, with the command "getconf CLK_TCK" which again on this system gives you 100.

We call this value "Hertz" and is usually 100 or 1024 though for some architectures it is other values.

Start Time
For each process the procfs has the start time, but it is expressed as the number of jiffes since the computer was booted.  If we want to know the real "wall clock" time a process is started we have to start with number of seconds since epoch (for example using the time() function) subtract the number of seconds since the computer was booted and add the process start time from procfs.

The seconds since boot is the first value in /proc/uptime:
696141.85 1371857.53

The 22nd item in a procfs PID stat file is the number of jiffes since boot when the process started. We have to divide it by Hertz to calculate the number of seconds.
27139 (bash) S 11942 27139 27139 34821 5200 4202496 5500 25537 0 14 40 16 18 15 20 0 1 0 69446639 23433216 1378 18446744073709551615 4194304 5082140 14073724008 6512 140737240085456 140127945249678 0 65536 3686404 1266761467 0 0 0 17 1 0 0 0 0 0

To get the start time, take the current time, subtract the boot time and add the process start time.
perl -e '$nowstr = localtime(time() - 696141.85 + 69446639 / 100); print $nowstr, "\n";
Mon Jul 12 22:20:07 2010

Unless you are very quick, you will have to take a few seconds off because the time() function will be now, while the values from the proc files will be a few seconds earlier as it takes time for you to cut and paste.


1
]]> Book: Do Androids Dream of Electric Sheep? tag:www.enc.com.au,2010:/sees-blog//1.27 2010-06-22T10:32:52Z 2010-06-22T10:56:13Z Actually they're more interested in freedom, to be no longer slaves to humans. I don't think it's any spoiler to mention that an electric sheep does appear in the book.This is a classic science fiction story of a dark not-so-distant... Craig electric sheep does appear in the book.

This is a classic science fiction story of a dark not-so-distant future.  In fact given that it's over 40 years old it could be set in some alternate now.  Some disaster has occurred and many have either left earth or died.  Some andriods, or andys have escaped and bounty hunters are after them.

You may not know the book by its correct title, but you may of heard the movie that is based upon it, called Blade Runner.  Like most movies based on books, it takes some of the concepts and ideas of the book but in my opinion is a pale imitation.

I'm glad I read this book. It hasn't aged that much and it's very though provoking.  Sci-Fi often uses some alternate reality to put a mirror up to what happens in the real world and this book is no exception.

And a token link to Debian, if you want you own electric sheep you can install the screensaver electricsheep package which draw pretty fractals on your screen.

Related articles by Zemanta
]]> Helping Free Software: Translations tag:www.enc.com.au,2010:/sees-blog//1.26 2010-06-20T03:32:13Z 2010-06-20T04:04:27Z Well the latest version of PSmisc is almost finished, so I pushed the tar archive up to the spot for the translators, waiting for the updates before it gets released.It reminded me that perhaps a lot of people don't know... Craig PSmisc is almost finished, so I pushed the tar archive up to the spot for the translators, waiting for the updates before it gets released.

It reminded me that perhaps a lot of people don't know how their programs get translated. Perhaps there are some that could even contribute.  The nice thing about the translation systems is that you don't need to know programming to help.

PSmisc uses the translation project as its method.  The program itself uses gettext for the translation part which makes a single text file called a POT file.  A translator takes this text file and makes a po file, such as it.po for the Italian translation and then translates, which basically means editing the file, reading each first line of a set and writing the translation on the second line.

For the programmer, once it is setup it is pretty simple to use, just mark your translatable lines and follow some simple rules, mainly about not embedding too much in a string.  I've used this system for years for quite a few of my programs and there is little added work for me.

Unfortunately,  translating is hard work and should be a long-term commitment.  Having a look at the translation project's Translation Matrix and you can see that some languages do suffer, though there are some very good results there too.  Particular kudos goes to the Vietnamese group which I think is just Clytie who does a marvellous job and is often the first translation file I get.

For the Debian project, there are many places where translations can occur.  There doesn't seem to be a centralised place for this, but some of the places they use translation work are:

]]> Australian Internet Censorship and Data Retention tag:www.enc.com.au,2010:/sees-blog//1.25 2010-06-15T10:52:48Z 2010-06-15T11:30:15Z Craig Australian government is trying to beat the previous government's poor record.  Where the previous government seemed to think the internet a scary and unimportant thing that they didn't really understand and therefore ignored it, the current government seems to be trying to do something, but like a lot of other things they do, do something badly.

Internet Filtering
One of their bright ideas is to censor the internet, by putting some rather large filters right in the middle of all the ISPs.  Supposedly this is going to protect the children, though other than some mad ranting by Senator Conroy he hasn't said which children or how.

The clearest information is that it will block Refused Content or RC rated information.  The problem with this is there is no clear definition what this is.  With no clear boundary you can get what is "scope creep".  Bit by bit, each group with their own agenda will try to get whatever they don't like banned.  Some will fail, but others will get their little set of demons onto the list.

From "children overboard" to the strange siezure of the wikileaks founder's passport when he returned to Australia, you can never trust the government fully. As the filter list will be a closed list, who is to say if it is right a particular webpage or website should be banned?  Books or films that are banned are known, you can find out what they are and why.  A proper discussion and review can then be undertaken.  By contrast, you won't even know something is banned unless you try to visit it.

I've personally seen the "great firewall of china".  What is filtered is often arbitary, though anything that is embarassing to the government is filtered.  It slows a lot of sites down and makes others look strange.  Do you really what to live in a country where the government decides what ideas should be seen?  Or even a country that places like China can point to and say they are doing the same thing, so its all ok?

Data Retention
The next bright idea by the government is to make ISPs keep 10 years of internet browsing history of all their users.  This would be like tapping everyone's phone, just in case you did something wrong in the next 10 years.

There hasn't been much detail about this proposal but let's assume for a moment that it keeps URLs.  Now of course most people's internet addresses move around, so you will also need to keep some sort of log of which account used what address for the same time.

The URLs will tell the government which websites you have visited, but URLs also tell them which pages you visited.  You can also often assume which pages you read and which you didn't because of the time between this viewed page and the next.

Search engine queries are also encoded into a url. Google searches usually have something at the end of the url which is what you were searching for.  There is also a chain of visited pages, so someone looking at a log could see your search, you go to a site, perhaps you then visit a banking or paypal site (have you bought something now?)

Even if you think you have nothing to worry about what the government might do with this information, including future governments, this information has to sit somewhere.  Data sitting around for 10 years has 10 years time of being stolen or copied.  Perhaps some activists obtain this log and publish a list of names of people who visited a particular website.

There is current laws for lawful interception.  This is where the police or another security agency goes to the court and says a particular person has done certain bad things and asks if they can intercept their internet traffic.  It's the same rules if they want to tap your phone.  Except for "fishing trips" where police just randomly look at information from anyone hoping to trip up on something, what is this system going to be used for?

What can you do?
If you're not happy about either, or both, of these new proposals, it is time to do something about it. Visit the website Open Internet Website for what you can do.  One of the things is to
tell you mum about internet censorship which has a funny message from comedian Akmahl Saleh that also has an important side.

Most of important of all, don't let these proposals become law because of apathy!

Related articles by Zemanta
]]> Does your program use gethostbyname() ? tag:www.enc.com.au,2010:/sees-blog//1.24 2010-06-13T04:30:32Z 2010-06-13T04:46:07Z Craig Debian IPv6 list about how the function gethostbyname() has changed.  The manual page says this function is obsolete and with good reason.

A new feature of the function is that if a hostname has both IPv4 and IPv6 entries then the order of the return values is not predefined.  In fact it used to be you'd get the IPv4 addresses first, then the IPv6.  That has now changed and with the more recent glibc you will get an IPv6 address first.  Quite often old code doesn't even check the address family or the size of address structure but just copies the relevant 4 bytes.  You then get the first part of an 16-byte IPv6 address wedged into a 4-byte IPv4 address structure which basically means you get a mess.

The fix is pretty simple by changing the system call from gethostbyname() and using getaddrinfo() instead.  If you only want the IPv4 addresses, perhaps because other parts of the code have not been checked or changed, then set the hints variable to return only IPv4 addresses.

If one of your packages is starting to play up and unable to connect to certain remote places and is instead trying to get to some "random" address, have a quick check for gethostbyname(). A quick grep across the source code may save a lot of debugging time.

 




]]> JFFNMS 0.8.5 released tag:www.enc.com.au,2010:/sees-blog//1.23 2010-05-09T10:40:28Z 2010-05-09T10:45:01Z Craig PHP and database exports, jffnms 0.8.5 is now released. This program is a network management system written in PHP.  The worst part about the whole maintaining process for it would have to be is the release.

Why is it so difficult to track changes within a database and PHP code? You don't get that nice compile-time versus run-time error problems and the database is just diabolical to keep up to date with what you have changed.  Someone needs to invent a git for database states!

Looking around other PHP based programs, I don't think anyone else has solved this issue. Well, its out there, enjoy it or not and if you have comments about the program let me know.
]]> VMware at last tag:www.enc.com.au,2010:/sees-blog//1.22 2010-03-12T10:56:06Z 2010-03-12T11:09:04Z Craig VMware, but its an important piece of software at my workplace, it also allows me to run some important VMKs or modules.  But at first it just wouldn't compile.

And then I found this wonderful blog about Installing VMware on Ubuntu and it worked wonderfully. The author patched the code and it all installed nicely.

The next problem was one of the services would not start. Port 8308 would refuse to work and when I went to the management screen and said Service Unavailable.  I tracked this down to the Java program dying at socket binding time.

The absolute first thing you should check if you are having TCP/IP problems with Java is the sysctl path net.ipv6.bindv6only which you can check with
sysctl net.ipv6.bindv6only.

If it is 1, it might mean bad Java code network problems. And in fact this time it was the problem, changing it to 0 and the Java daemon started and stayed running and all was good.

Incidently if you use the Cisco ASA firewall Java client and it dies, use this trick for it too.  On Debian systems, edit the file /etc/sysctl.d/bindv6only and set that option to 0.  I don't think its the fault of the key, but bad Java code (but is there anything but bad Java code?)
]]> dh-make updated to 0.52 and its remaining bugs tag:www.enc.com.au,2010:/sees-blog//1.21 2010-03-08T02:49:19Z 2010-03-08T03:07:00Z Craig
If you get weird lintian errors about native source formats and version numbers wrong on your brand-new Debian package you just made, you might of been bitten by this bug.

That leaves a few bugs left, there are two I need help with:
  • Bug 328692 - If you have a plan ASCII name like me, then your name looks fine in the dh-make generated man pages. This bug requires converting names that aren't plain ASCII (e.g. that use UTF-8) into something groff understands. Someone suggested decomposed unicode but unicode and groff are pretty much a black art to me, let alone combining them.
  • Bug 533117 - This one is all about making dh-make make your python packages.  You'll need to understand the new debhelper v7 rules files.  Again, python is not something I use so the bug is stuck here.
Related articles by Zemanta
]]> Gjay Updated tag:www.enc.com.au,2010:/sees-blog//1.20 2010-03-07T11:05:40Z 2010-03-07T11:09:42Z Craig Gjay 0.3.0 is released.  This is my first release of Chuck Groom's code and hopefully it will work for you too.

The Debian packages will be out shortly after some building and testing.  If you have a 64-bit computer it now works with 64 bits fine (ie on my amd64).

It still needs some work, I'd like it to interact with more than audacious as the sound player. Also if you know how to in one of the sound libraries stream wav, ogg or mp3 files correctly I'd like to hear from you.  Currently gjay just uses the same old fork to mpg321 method, but idealy I'd like it to use the libraries directly.

]]> procps: Third time lucky tag:www.enc.com.au,2010:/sees-blog//1.19 2010-02-02T11:41:00Z 2010-02-02T12:10:37Z OK, ok, i got a chroot and pbuilder now. So that should, I hope stop any more FTBFS bugs about missing depdendencies.procps got uploaded that fixes some important bugs, but mainly they were small fiddly things. About the most significant... Craig pbuilder now. So that should, I hope stop any more FTBFS bugs about missing depdendencies.

procps got uploaded that fixes some important bugs, but mainly they were small fiddly things. About the most significant enhancement was pmap now has a real working -x flag.  It looks a lot like some of the other pmap programs out there and shows the RSS and Dirty bytes per map. Let me know if its useful or not.

However there still is 48 bugs in the package, so if you're feeling game wander over to procps bug page and have a look around, but here are some more interesting ones, such as why would a process start time be earlier than the boot time? Bug 408879 has this problem

Now, a nice can of worms is in a Linux system, what is free memory?  What should the "free" program report?  Currently free just reports what it sees in /proc, but in Bug 565518 should the slab count be taken out?  I certainly won't be making any Debian-specific changes here as you could get different numbers depending on your distribution, or even worse the age of you Debian system.

procps is also my first attempt at using git-buildpackage which I found very helpful. There was one problem with it and that is how it works with the quilt patch program. If the quilt patches are applied, git doesn't know this and says all the files have changed. I know its how these two programs are supposed to work but its a little annoying.
]]> The Great Australian Internet Firewall tag:www.enc.com.au,2010:/sees-blog//1.18 2010-01-28T23:49:54Z 2010-01-29T00:24:15Z Craig T he Great Australian Internet Blackout which is a protest about the upcoming Australian Governments Internet Filtering.

As per usual with this sort of thing, the proposed solution will not even meet its rather vague goals, unless you count filtering a 1000 or so websites a "goal".  While the numbers vary, the best estimates is its way over 100 to 200 million websites. So that 1000 is 0.0005, or 1 in 200,000.

The filter is alternatively going to filter reported refused classification material or perhaps "other stuff"; they'll get back to you on the second one. For the first it works that somone finds a website of sufficient level of classification, reports it to the government and it gets filtered.  And how many times have you come across RC websites, just by wandering around? That's not R rated websites, its not porn, its not even reasonably exotic porn, but stuff that's downright nasty.

The government, as per usual, has come out with the tired old protect the children rant. The filter doesn't stop spam, it doesn't stop weirdos on facebook or MSN chatting up your teenage daughter or son, it filters websites that, unless you or your child is REAL unlucky you will never see; ever.

So why am I against it? Surely filtering RC level websites is great right? And to be honest if that is all this ever was going to be, it's still a complete waste of my (and if you are an Australian Internet user) your money, but once the filtering is in, there is going to be a line outside the relevant ministers door to filter anything else anyone wants to filter, for example:
  • Games that are available outside Australia, if they have not been classified here then they shouldn't be available here
  • Torrent trackers, because the industry would no doubt tell the government that the only thing they're good for is illegal download of movies or music
  • Websites about certain topics that some people would rather not be seen, there is a large mountain of stuff that fits in here depending which crank has their own barrow to push
This sort of filtering while initially seeming ok, will get worse, much much worse.  There is not a real strong push from the general public about filtering RC websites, most people see and hate spam much more.  If you are an Australian citizen I ask you to visit the The Great Australian Internet Firewall  website and write to your local member of parliament.

Related articles by Zemanta
]]> Manilla, Git and Gjay tag:www.enc.com.au,2010:/sees-blog//1.17 2010-01-27T23:30:21Z 2010-01-27T23:57:46Z Craig Philippines, but here I am.  It's a reasonably modern place and to me feels more like America than Asia in so many ways, posssibly because of its history.  One thing is for sure, noone follows road rules here.  Red lights are a suggestion and a zebra crossing is just some painted lines that you do need to stop at.

As for food, it's not that different, in fact this sad lot is about what was different:
  • McSpaghetti - Was supposedly sugar coated spaghetti but actually was very tame, my son would of loved it.
  • Wow Steak from KFC - Neither Wow nor Steak, like a big chicken nugget with gravy and rice
  • Halo Halo - A dessert drink which was sugared or preserved fruit, milk and ice.
Generally though the food has been pretty good but nothing I could get at home.

Good news about Gjay, the previous maintainer said it was ok for SourceForge to hand over the control of the program to me so I've set it up in git and started working on it.  Most of the work was getting the code up-to-date to the later gtk APIs and making it work with audacious instead of xmms.  It's almost ready for (re)initial release and there is even an ITP ready to go.

Git is a rather interesting and new (for me at least) version control system.  I've been using cvs for more years than I'd care to think about and svn but while it is a bit different as you'd expect I haven't had it get in my way.  In fact I've been so impressed with Git I have put a few other projects into it, mainly with the collab maintence Debian project for a few of my packages.

]]> Happy new RC tag:www.enc.com.au,2010:/sees-blog//1.5 2010-01-01T06:48:53Z 2010-01-01T13:12:58Z Craig
After a lot of waiting, JFFNMS release candidate 1 for 0.8.5 got uploaded to sourceforge.  This release is mainly about fixing some database release bugs 0.8.4 had and they're all caused by the fact that working with PHP and database to release code is plain awful.

The problem is tracking changes in your database. So version 1 has 3 tables and 60 rows, version 2 has 4 tables and 90 rows, but what changed?  Everything I've seen so far is a bit of a hack or is real fiddly.  JFFNMS release process is both which is why I'll go and release several versions of C code or Debian packages before trying to crack that nut again.

If you are wondering what JFFNMS is, its a Network Management System. It makes graphs and red/green icons depending on the status of your routers and servers. Written in PHP and web based and of course licensed under the GPL
]]> Updated: psmisc, gw6c and gjay tag:www.enc.com.au,2009:/sees-blog//1.4 2009-12-29T07:30:46Z 2009-12-29T11:52:33Z Craig
psmisc got a bunch of updates, including a new program called prtstat which formats the stat file in the procfs for a pid in (hopefully) a nice way.  No sooner had I released the latest update when a bug report came in. It seems fuser -m -k is a little too happy about killing itself. The fix is in the CVS but anoying I missed that.

Next up was the Debian gw6c package. I was asked why it didn't get moved from unstable to testing.  The problem is that while Linux has iproute, kfreebsd does not so the lack of a dependency was stopping it transitioning.  To make matters worse the freebsd template was missing from their package.  After some deb-substvars evilness to fix the dependencies and some dh_install overrides in the debian/rules file it should all be happy when its finished.

Finally, I miss having good random playlists. I'm too lazy to make them myself so I use some random thing which often gives me rubbish.  A program called Gjay used to be in the Debian archive but got removed, mainly because the upstream stopped supporting it.  I can write C (the programming language its written in) and I wanted to use it, so I fixed it.  My version is 64-bit clean, so it works on my amd64 and it works with audacious not the old xmms which is great.  More importantly, it compiles, it runs and it even works properly.
I'm just wondering if I want to release it out to the wider world or not.
]]> Why clocks don't work in PNG tag:www.enc.com.au,2009:/sees-blog//1.3 2009-12-19T06:53:48Z 2009-12-29T11:53:04Z Craig Port Moresby, Papua New Guinea. Lots of stuff doesn't work too well over there. It's hard to get experience maintenance staff, its hard to get the right parts over there and probably the hardest part is paying for it. 

I was always surprised the clocks never work right. They are either horribly fast or slow.  At first I thought the hotel had just not set it right, so I set it to the right time. But the next morning it was out again. What the hell was going on?

Most alarm clocks work off the power. In both Australia and PNG this is 50Hz, so the clock rectifies the AC power coming in to get 50 pulses a second. Count 50 pulses and you have a second, do this 60 times and tick over a minute.

We take for granted that the frquency of power is right on 50Hz, but talking with the students who were doing power engineering it is actually quite hard to do. It's like having a car with the same revs no matter if you are going up or down a hill.

Anyhow, if you need to wake up the right time in PNG, use the alarm clock on your phone. Otherwise you may be jumping out of bed at 5am thinking you are running late, then wondering where the sun went.
]]>