tag:blogger.com,1999:blog-63902798529456122462024-03-18T03:03:48.703+00:00Handy Linux notesLinux notes, tips and tricks. Handy scripts and solutions to arcane problems.Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.comBlogger19125tag:blogger.com,1999:blog-6390279852945612246.post-90918441290669199642013-07-06T22:50:00.000+01:002013-07-06T22:50:43.321+01:00Get-iplayer: The solution to the ERROR: RTMP_ReadPacket problem<div dir="ltr" style="text-align: left;" trbidi="on">
If you have installed (or reinstalled) get-iplayer and it not longer works, but throw something like this:<br />
<br />
<blockquote class="tr_bq">
Connecting ...<br />INFO: Connected...<br />ERROR: RTMP_ReadPacket, failed to read RTMP packet header<br />INFO: Command exit code 1 (raw code = 256)<br />WARNING: Failed to stream file [some program file].partial.mp4.flv via RTMP<br />INFO: skipping flashstd2 mode<br />ERROR: Failed to record '[the program name]'</blockquote>
The solution is to run this command:<br />
<blockquote class="tr_bq">
<span style="color: black;"><span style="font-family: arial;">get_iplayer
--prefs-add
--rtmp-tv-opts="--swfVfy=http://www.bbc.co.uk/emp/releases/iplayer/revisions/617463_618125_4/617463_618125_4_emp.swf"</span></span></blockquote>
If that doesn't work, make sure that you have <i>lame,</i> <i>ffmpeg, RTMPdump </i>and<i> flvstreamer</i> installed.</div>
Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-5345359567945418242013-06-15T20:38:00.000+01:002013-06-15T20:38:35.503+01:00Temporarily disabling the touchpad when typingTired of typing and accidentally changing the cursor position? A little known trick in KDE enables you to disable the touchpad. It comes back after a configured amount of time.<br />
<br />
1) Install kde-config-touchpad<br />
2) K -> Applications -> Utilities -> Touchpad Management<br />
<br />
Now you are able to configure the touchpad. You can select "Automatically turn off touchpad on keyboard activity".<br />
<br />
This means that whenever you are typing, touching the touchpad will no longer cause you problems. After a short delay, once you have stopped typing, the touchpad will return into service.<br />
<br />Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com3tag:blogger.com,1999:blog-6390279852945612246.post-57934092504829088142013-01-30T19:58:00.001+00:002013-01-30T19:59:58.139+00:00Thunderbird, Apache, Wevdav and Filelink: Rolling your own Thunderbird filelink<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: center;">
Thunderbird, Apache, Filelink</h2>
<div style="text-align: left;">
If you are sending an email with a large file attachment, the chances are that it will bounce back with a message saying that the email exceeds the maximum allowed.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Emailing large documents, presentations, images and other multimedia content is the norm these days. This means that bounced emails are a regular plague.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Thunderbird has solved this problem by implementing "Filelink" technology. This works by uploading the attachment to a server and converting the attachment into an embedded link in the email.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Several services are presently included in the Thunderbird offering, including <a href="https://one.ubuntu.com/" target="_blank">Ubuntu One</a>, <a href="https://www.yousendit.com/" target="_blank">YouSendIt</a> and <a href="https://www.box.com/" target="_blank">Box</a>. The Mozilla page for filelink is here: <a href="https://support.mozillamessaging.com/en-US/kb/filelink-large-attachments" target="_blank">Filelink</a>.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
However, most users are unlikely to want to trust their attachments to a third party service. There is a way to host your own solution, using Apache and turning on WebDav. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
First, you need to install the Thunderbird addon, <a href="https://addons.mozilla.org/thunderbird/addon/webdav-for-filelink" target="_blank">Webdav for filelink</a>.</div>
<ol style="text-align: left;">
<li>Go to <a href="https://addons.mozilla.org/thunderbird/addon/webdav-for-filelink" target="_blank">Webdav for filelink</a> and install the add-on in Thunderbird (alternatively, Tools->Addons in Thunderbird and search for it in there and install it).</li>
<li>After you have configured your webdav service in Apache, you will need to configure <a href="https://addons.mozilla.org/thunderbird/addon/webdav-for-filelink" target="_blank">Webdav for filelink</a> so that it knows which server to log in to and how to authenticate.</li>
</ol>
Now you need to set up your Apache server so that it supports webdav. Of course you are running Linux.<br />
<br />
These days, Apache on Linux usually has the directory /etc/httpd/conf.d. So in this directory, create the file webdav.conf with this content:<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> <ifmodule mod_dav.c=""><br /> LimitXMLRequestBody 50000000<br /><br /> Alias /webdav "/home/httpd/webdav"<br /> <directory home="" httpd="" webdav=""><br /> Dav On<br /> Options +Indexes<br /> IndexOptions FancyIndexing<br /> AddDefaultCharset UTF-8<br /> AuthType Basic<br /> AuthName "WebDAV Server"<br /> AuthUserFile /etc/httpd/webdav.users.pwd<br /> Require valid-user<br /> Order allow,deny<br /> Allow from all<br /> </directory><br /> </ifmodule></span></span></blockquote>
Otherwise, if you don't have the directory, then put that content into your httpd.conf file. <i>httpd.conf</i> is often found in the <i>conf</i> directory.<br />
<br />
Note that you may choose to have your webdav directory something other than our setting (<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/home/httpd/webdav</span></span>). You<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span> will need to create this directory:<br />
<ol style="text-align: left;">
<li>mkdir <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/home/httpd/webdav</span></span></li>
<li><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span>chown apache <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/home/httpd/webdav</span></span></li>
<li>chgrp apache <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/home/httpd/webdav</span></span></li>
<li><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span>chmod 700 <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/home/httpd/webdav</span></span></li>
</ol>
for <i>apache</i> use the user and group name for your apache process.<br />
<br />
Now you need to create the password file for authentication to the webdav service:<br />
<ol style="text-align: left;">
<li>htpasswd -c /etc/httpd/webdav.users.pwd myuser</li>
<li>Repeat (1) for all the users that you which to grant access to, without the "-c" flag.</li>
</ol>
where <i>myuser</i> is the user you are giving access to the service to, probably the first time, your own username.<br />
<br />
Open your httpd.conf. Check that it has these items in it:<br />
<ul style="text-align: left;">
<li>LoadModule dav_module modules/mod_dav.so </li>
<li>LoadModule dav_fs_module modules/mod_dav_fs.so</li>
<li><blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><ifmodule mod_dav_fs.c=""><br /> # Location of the WebDAV lock database.<br /> DAVLockDB /var/lib/dav/lockdb</ifmodule></span></span></blockquote>
</li>
</ul>
Now run:<br />
<ul style="text-align: left;">
<li>/etc/init.d/httpd configtest</li>
<li>if all is okay: /etc/init.d/httpd restart</li>
</ul>
You are ready to test your webdav. In firefox, go to:<br />
<blockquote class="tr_bq">
http://mydomain.com/webdav</blockquote>
(of course you substitute your own domain name for "mydomain.com"!).<br />
<br />
You should be prompted to for a username and password, enter the credentials that you created above (in the <i>htpasswd</i> bit).<br />
<br />
If that works, then you are nearly there:<br />
<ol style="text-align: left;">
<li>Open up Thunderbird</li>
<li>Edit->Preferences->Attachments->Add</li>
<li>Choose "Webdav"</li>
<li>In <i>location</i> enter: http://mydomain.com/webdav</li>
<li>Enter your authentication and choose "remember password" when prompted. </li>
<li>Click OK and you're done </li>
</ol>
Thunderbird checks that it can authenticate on the webdav service immediatley. The actual verification should be almost immediate. There are two possible points of failure in the above:<br />
<ol style="text-align: left;">
<li>You entered the wrong username/password or the server entry isn't correct. This will generate authentication or server error messages in Thunderbird.</li>
<li>The URL is recognised by Apache but it isn't the proper webdav URL. When this happens, Thunderbird tries and tries again for a long time, ultimately failing to authenticate. </li>
</ol>
<br />
From now on, when you create an email and put in one of more large attachments, Thunderbird will ask you if you wish to convert them into a link instead. You can also right-click on an attachment, choose "covert to" and then choose link.<br />
<b><br /></b>
<b>NOTE</b>: If, when you are composing your first email using a large attachment, your Thunderbird filelink keeps failing with "Webdav failed authentication", the chances are that your permissions are not correct for apache to write to the webdav directory on the server that you specified. Check your permissions! Thunderbird has already verified the logging in, so this is actually the wrong message being reported. Unfortunately, you probably won't see these permissions failures in the log files on the server either.<br />
<br /></div>
Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com1tag:blogger.com,1999:blog-6390279852945612246.post-3186042998875908962013-01-17T15:46:00.002+00:002013-01-17T15:46:40.240+00:00Oracle to Postgres conversion tool<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: left;">
<span style="font-family: Times,"Times New Roman",serif;"> Do you have Oracle databases that you wish to convert to Postgres? A new version of the Oracle to Postgres conversion too, Ora2Pg, has been released.</span></div>
<div style="text-align: left;">
<span style="font-family: Times,"Times New Roman",serif;"><br /></span></div>
<ul style="text-align: left;">
<li><div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;">Website: <a class="moz-txt-link-freetext" href="http://ora2pg.darold.net/">http://ora2pg.darold.net/</a> </span></div>
</li>
<li><div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;">Download: <a class="moz-txt-link-freetext" href="http://sourceforge.net/projects/ora2pg/">http://sourceforge.net/projects/ora2pg/</a></span></div>
</li>
<li><div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;">Development: <a class="moz-txt-link-freetext" href="https://github.com/darold/ora2pg">https://github.com/darold/ora2pg</a> </span></div>
</li>
<li><div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;">Changelog: <a class="moz-txt-link-freetext" href="https://github.com/darold/ora2pg/blob/master/changelog">https://github.com/darold/ora2pg/blob/master/changelog</a> </span></div>
</li>
</ul>
<div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;"><b>About Ora2Pg</b> </span></div>
<div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;">Ora2Pg is an easy and reliable tool to migrate from Oracle to
PostgreSQL. It is developed since 2001 and can export most of the Oracle
objects (table, view, tablespace, sequence, indexes, trigger, grant,
function, procedure, package, partition, data, blob and external table).
Ora2Pg works on any platform and is available under the GPL v3 licence.
Docs, Download & Support at <a class="moz-txt-link-freetext" href="http://ora2pg.darold.net/">http://ora2pg.darold.net/</a></span></div>
<div style="text-align: left;" wrap="">
<br /></div>
<div style="text-align: left;" wrap="">
<span style="font-family: Times,"Times New Roman",serif;"><b>About Postgresql</b></span></div>
<div style="text-align: left;">
<span style="font-family: Times,"Times New Roman",serif;"> Postgresql (Postgres) is an enterprise class database system (DBMS) that is OpenSource, free to use and free of licence fees, that is released under the PostgreSQL License, which is an <a href="http://en.wikipedia.org/wiki/MIT_License" title="MIT License">MIT-style license.</a></span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: Times,"Times New Roman",serif;">It compares very favourably to enterprise grade databases such as <a href="http://www.oracle.com/" target="_blank">Oracle</a>, <a href="http://www.actian.com/" target="_blank">Ingres (actian)</a> and <a href="http://www-01.ibm.com/software/data/db2/" target="_blank">DB2</a>. It is an upgrade from mid-range database products and light weight / embedded data stores.</span></div>
<div style="text-align: left;">
<span style="font-family: Times,"Times New Roman",serif;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: Times,"Times New Roman",serif;">Postgres is actively developed and has a large, active online community with many companies (including Kieser.net) offering commercial Postgres support.</span></div>
<div style="text-align: left;" wrap="">
</div>
</div>
Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com2tag:blogger.com,1999:blog-6390279852945612246.post-85773734062529438122010-09-16T16:35:00.000+01:002010-09-16T16:35:57.465+01:00Converting PST files to Linux MBOX formatYou want to upgrade from Outlook to Thunderbird and you've changed from Windows to Linux as a desktop. If you have control over a mail server running dovecot, you are in luck.<br />
<br />
The tool that you need is the very handy <a href="http://www.five-ten-sg.com/libpst/">libpst</a> library, although if you run Kubuntu or Ubuntu then this is available on synaptic.<br />
<br />
Simply export your Outlook mail into a pst file. Let's call the file mailBackup.pst. Now copy the file to your email server or linux desktop.<br />
<br />
It is important to create a directory to work in because for each folder in the PST (Outlook) file, there will be a mbox file created with the same name. Cd into this directory and keep emailBackup.pst in the directory above it.<br />
<br />
Now simply run:<br />
readpst ../emailBackup.pst<br />
<br />
The Outlook file will be converted into a series of mbox files. If you run Dovecot, these can be put into your mail directory (be careful not to overwrite existing files!) and you will be able to access them with Thunderbird or your webmail software (I recommend <a href="http://www.horde.org/">horde</a>).<br />
<br />
You are also able to open mbox files in Thunderbird.Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com37tag:blogger.com,1999:blog-6390279852945612246.post-7704207227654538932010-06-03T19:13:00.001+01:002010-06-03T19:14:40.993+01:00SpamAssassin (spamd) deadlocks, runnning slow, PostgresThe usual way to use spamassassin is to implement the Bayesian filtering database in flat files. That's the default configuration and most distros ship with this option.<br />
<br />
But that's a bad option. If you want to implement a database, use a proper database.<br />
<br />
The advantages are numerous: Proper caching and cache management, ability to tune the database and tables, better locking capabilities (as in the locking capabilities exist), clustering and fail-over capabilities which all amount to better resource usage, better throughput, less server strain, better maintenance and a much better service. <br />
<br />
There is no better enterprise class OpenSource database than Postgres (or it's twin, Ingres). Frankly, your decision to NOT use Postgres for your database needs has to be very well justified as PG is not only truly enterprise class, but it's also easy to set up, easy to admin and, most importantly. you can tune it properly.<br />
<br />
Which is why you should be using Postgres with SpamAssassin.<br />
<br />
Unfortunately, though, the latest version (and previous versions), 3.3.1 have some pretty bad SQL in them. Rather than utilise the PG strengths and keys, the SQL has not been optimised from a performance point of view. Which, in an email system, is one of the most important things to consider!<br />
<br />
The biggest hole is the use of the SQL I<i>N</i> operator on the bayes_token table. This effectively forces a full table scan because the unique key is id, token. On a system-wide implementation, the ID column is a particularly weak key (i.e. not a key at all because it's always the same value) so this is a real deal-breaker.<br />
<br />
The solution is to use the primary key wherever possible, which, it turns out, is nearly all the time.<br />
<br />
On a system with a large spam database, this is the difference between a powerful server grinding to its knees v.s. the same server flying at vast throughput.<br />
<br />
The biggest deal-breaker is in the update of the atime column, which is about the most regularly performed task. So it's the hottest of the hot spots in the spamd PG code and also the worst implemented. The fix, however, is very easy.<br />
<br />
Simply edit this file (note the path will be different on your machine:<br />
/usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/BayesStore/PgSQL.pm<br />
<br />
and make these changes:<br />
<br />
<br />
<i><b>Original code fragment</b></i>:<br />
------------- <br />
<div style="font-family: Arial,Helvetica,sans-serif;">sub tok_touch_all { </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;"> my $sql = "UPDATE bayes_token SET atime = ? WHERE id = ? AND token IN ("; </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my @bindings; </div><div style="font-family: Arial,Helvetica,sans-serif;"> foreach my $token (sort @{$tokens}) { </div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql .= "?,"; </div><div style="font-family: Arial,Helvetica,sans-serif;"> push(@bindings, $token); </div><div style="font-family: Arial,Helvetica,sans-serif;"> } </div><div style="font-family: Arial,Helvetica,sans-serif;"> chop($sql); # get rid of trailing , </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql .= ") AND atime < ?"; </div><br />
-------------- <br />
<br />
<i><br />
</i> <br />
<i>Amendments:</i><br />
---------- <br />
<br />
<div style="font-family: Arial,Helvetica,sans-serif;">sub tok_touch_all { </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> foreach my $token (sort @{$tokens}) { </div><div style="font-family: Arial,Helvetica,sans-serif;"> my $sql = "UPDATE bayes_token SET atime = ? WHERE id = ? AND token ="; </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my @bindings; </div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql .= "?,"; </div><div style="font-family: Arial,Helvetica,sans-serif;"> push(@bindings, $token); </div><div style="font-family: Arial,Helvetica,sans-serif;"> chop($sql); # get rid of trailing , </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql .= " AND atime < ?"; </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;">. </div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->commit(); </div><div style="font-family: Arial,Helvetica,sans-serif;"> } </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 1; </div><div style="font-family: Arial,Helvetica,sans-serif;">} </div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div>---------------------------- <br />
<br />
Note that I insert the closing } before the "return 1". <br />
<br />
I.e. I have converted this into a line by line update, so that the DB can use the very strong primary key of id,token. <br />
<br />
The performance difference that this makes is absolutely enormous on a busy system. <br />
<br />
In case you want to simply cut and paste the entire function, here is the tok_touch_all function with the amendments in it:<br />
<br />
<div style="font-family: Arial,Helvetica,sans-serif;">sub tok_touch_all {</div><div style="font-family: Arial,Helvetica,sans-serif;"> my ($self, $tokens, $atime) = @_;</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 0 unless (defined($self->{_dbh}));</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 1 unless (scalar(@{$tokens}));</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> foreach my $token (sort @{$tokens}) {</div><div style="font-family: Arial,Helvetica,sans-serif;"> my $sql = "UPDATE bayes_token SET atime = ? WHERE id = ? AND token =";</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my @bindings;</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql .= "?,";</div><div style="font-family: Arial,Helvetica,sans-serif;"> push(@bindings, $token);</div><div style="font-family: Arial,Helvetica,sans-serif;"> chop($sql); # get rid of trailing ,</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql .= " AND atime < ?";</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->begin_work();</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my $sth = $self->{_dbh}->prepare_cached($sql);</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> unless (defined($sth)) {</div><div style="font-family: Arial,Helvetica,sans-serif;"> dbg("bayes: tok_touch_all: SQL error: ".$self->{_dbh}->errstr());</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->rollback();</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 0;</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my $bindcount = 1;</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sth->bind_param($bindcount++, $atime);</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sth->bind_param($bindcount++, $self->{_userid});</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> foreach my $binding (@bindings) {</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sth->bind_param($bindcount++, $binding, { pg_type => DBD::Pg::PG_BYTEA });</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sth->bind_param($bindcount, $atime);</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my $rc = $sth->execute();</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> unless ($rc) {</div><div style="font-family: Arial,Helvetica,sans-serif;"> dbg("bayes: tok_touch_all: SQL error: ".$self->{_dbh}->errstr());</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->rollback();</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 0;</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> my $rows = $sth->rows;</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> unless (defined($rows)) {</div><div style="font-family: Arial,Helvetica,sans-serif;"> dbg("bayes: tok_touch_all: SQL error: ".$self->{_dbh}->errstr());</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->rollback();</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 0;</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> # if we didn't update a row then no need to update newest_token_age</div><div style="font-family: Arial,Helvetica,sans-serif;"> if ($rows eq '0E0') {</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->commit();</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 1;</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> # need to check newest_token_age</div><div style="font-family: Arial,Helvetica,sans-serif;"> # no need to check oldest_token_age since we would only update if the</div><div style="font-family: Arial,Helvetica,sans-serif;"> # atime was newer than what is in the database</div><div style="font-family: Arial,Helvetica,sans-serif;"> $sql = "UPDATE bayes_vars</div><div style="font-family: Arial,Helvetica,sans-serif;"> SET newest_token_age = ?</div><div style="font-family: Arial,Helvetica,sans-serif;"> WHERE id = ?</div><div style="font-family: Arial,Helvetica,sans-serif;"> AND newest_token_age < ?";</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $rows = $self->{_dbh}->do($sql, undef, $atime, $self->{_userid}, $atime);</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> unless (defined($rows)) {</div><div style="font-family: Arial,Helvetica,sans-serif;"> dbg("bayes: tok_touch_all: SQL error: ".$self->{_dbh}->errstr());</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->rollback();</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 0;</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> $self->{_dbh}->commit();</div><div style="font-family: Arial,Helvetica,sans-serif;"> }</div><div style="font-family: Arial,Helvetica,sans-serif;"><br />
</div><div style="font-family: Arial,Helvetica,sans-serif;"> return 1;</div><div style="font-family: Arial,Helvetica,sans-serif;">}</div>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com2tag:blogger.com,1999:blog-6390279852945612246.post-75275540856450946702010-03-29T20:43:00.001+01:002011-09-28T11:47:26.871+01:00No sound in flash on AMD 64 Kubuntu<div dir="ltr" style="text-align: left;" trbidi="on">The standard ubuntu/kubuntu installation breaks sound in flash. There are several reasons for this, the first that Kubuntu/Ubuntu uses pulse audio and flash (along with many other applications) doesn't play nicely with pulseaudio. However, PulseAudio problems is just the first of two issues. The second is that Adobe 32 bit flash cannot share audio devices and output sound in the shared environment. For that you need the 64 bit version.<br />
<br />
So the first port of call in Kubuntu is to go to system settings/multimedia/device preference/audio output/Music and bump PulseAudio to the preferred (top) device.<br />
<br />
Click on "test" to make sure that pulseaudio is actually working on your system. If you don't have a sound coming out, you need to first get pulseaudio working properly.<br />
<br />
Next, you need to download the 64bit version of flash from Adobe and you can do this here:<br />
http://labs.adobe.com/downloads/flashplayer10_64bit.html<br />
<br />
(note the above is for version 10, you may wish to check that it's still the latest 64bit version).<br />
<br />
Unzip the file. It gives you a libflashplayer.so file.<br />
<br />
Now you need to overwrite this for firefox (and other applications that use flash).<br />
<br />
For example in my system, in my user home directory, I have ~/.mozilla/plugins/<br />
This is the directory that firefox/mozilla/chrome loads flash from. So I copy the file into this directory, overwriting the existing 32 bit version.<br />
<br />
You may want to<br />
<br />
sudo find / -name libflashplayer.so -print<br />
<br />
to find out where this file is elsewhere on your system and update it there too.</div>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com1tag:blogger.com,1999:blog-6390279852945612246.post-58131792458257432822010-01-19T22:11:00.000+00:002010-01-19T22:11:50.839+00:00KNetworkManager and WEP encryptionThe KDE desktop environment is excellent. KDE is famous for its well integrated, powerful set of system utilities and applications. It's stable, fast and fully featured yet remains easy to use and highly customisable. In short, it rocks.<br />
<br />
KDE is so widely used and supported that there is usually more than one utility to do any given task. However, there is usually an official tool and then alternatives.<br />
<br />
<br />
For managing networks, especially wireless and bluetooth networks, the knetworkmanager utility is the official tool to use. However, although knetworkmanager is very promising and integrates nicely into the KDE bar and is well designed and thought out, at present it seems to not be able to handle WEP encryption very well. Many people have reported problems with knetworkmanager whereas other network managers work well with the same settings on the same wireless network and hardware.<br />
<br />
<br />
We tested knetworkmanager with kubuntu Heron and a 10 character ASCII WEP key. While WiFI Radar worked well, connected immediately and scanned the range of wifi networks accurately, on the same laptop (Dell XPS M1730) knetworkmanager simply was unable to connect to the same WiFI connection that WiFI Radar connected to. We ensured that kwallet had the correct key in it and to make extra sure we also tested in the config file mode that knetworkmanager offers (storing the WEP passphrase in unencrypted text format).<br />
<br />
<br />
In /var/log/syslog there were numerous lines with the following entries:<br />
<i> <br />
</i><br />
<i></i><br />
<i><center>wlan0: AP denied authentication (auth_alg=1 code=15)<br />
<br />
NetworkManager: <info> Old device 'wlan0' activating, won't change.<br />
<br />
wlan0: RX authentication from XX:XX:XX:XX:XX:XX (alg=1 transaction=4 status=15)<br />
<br />
wlan0: unexpected authentication frame (alg=1 transaction=2)<br />
<br />
wlan0: replying to auth challenge<br />
<br />
wlan0: authentication with AP XX:XX:XX:XX:XX:XX timed out<br />
<br />
</info></center> <br />
</i><br />
<br />
It seems that for some reason, whilst other network management tools are able to configure the WEP passphrase correctly, KNetworkManager cannot. However, when we tested on unprotected WiFI networks, KNetworkManager worked a treat, reinforcing the notion that it only struggles with encryption.<br />
<br />
We also discovered during testing that sometimes other WiFI tools such as WiFI Radar scanned and reported more WiFI networks in the same area with the same laptop at the same time than KNetworkManager did. To be fair, we tried several scanning interations, starting up WiFI Radar and then KNetworkManager alternatively to ensure that the laptop hardware could still see all the WiFIs in the area. Not only was WiFI Radar consistent in its reports, but KNetworkManager was inconsistent, sometimes reporting the same number of WiFI networks as WiFI Radar, other times not seeing several of the networks.<br />
<br />
<br />
Browsing around the 'net, it seems that some people have KNetworkManager working and others do not. So at least part of KNetworkManager is functional, however if you are roaming networks and encounter a wide range of passphrases and WiFI configurations, this means that for now, KNetworkManager is practically unusable.<br />
<br />
<br />
So for now, it seems that unfortunately the KDE default network management tool should not be used. Instead, we would recommend that you try other tools. We found <a href="http://wifi-radar.systemimager.org/">WiFI Radar to be excellent</a>.However other tools are also available.Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com2tag:blogger.com,1999:blog-6390279852945612246.post-47380590675929718272010-01-19T22:07:00.000+00:002010-01-19T22:07:43.674+00:00Spamassassin tips and tricksSpamassassin is a powerful antispam tool. However, it consumes a lot of processing power, so a good idea is to install <a href="http://www.amavis.org/">amavis</a>. This is a lightweight Perl script that pre-scans emails and rejects many of them based on rules that you set up within the Amaviz configuration file.<br />
<i><br />
</i><br />
<i>NOTE</i>: <i>This page won't attempt to teach you how to install and configure Spamassassin or Amavis. other tutorials exist online. This tutorial is here to give you tips that you may not find elsewhere.</i><br />
<br />
Spamassassin uses bayesian filters (think of this as a form of artificial intelligence) that can learn about what sort of emails are spam (bad) and what sort are ham (good). The key to this is a tool called <i>sa-learn</i><span style="font-style: normal;"> which you run against mailbox files that either contain only ham or only spam emails. This allows Spamassassin to learn which emails you think are spam. Spamassassin uses several files to store this information, kept in a hidden directory (.spamassassin) for each mail user.</span><br />
<span style="font-style: normal;"><br />
</span><br />
<span style="font-style: normal;">To teach Spamassassin about spam, you pass the –-spam paramter to sa-learn. For ham, the parameter is –-ham.</span><br />
<span style="font-style: normal;"><br />
</span><br />
<span style="font-style: normal;">In the examples below we will assume that Spamassassin is running under the user account </span><i>spamd</i><span style="font-style: normal;"> and that a mailbox file (in the mbox format common with IMAP servers) that contains only sample spam emails is called </span><i>Junk</i><span style="font-style: normal;"> and is in the /tmp directory.</span><br />
<b><br />
</b><br />
<b>Tip 1</b>: Spamassassin with amavis uses the <span style="font-style: normal;">.spamassassindirectory in the Amavis working directory (usually<br />
/var/spool/amavis). Therefore when you are teaching Spamassassin called by Amavis, you need to use the --dbpath parameter. E.g.:</span><br />
<span style="font-style: normal;"><br />
</span><br />
<div style="margin-left: 1.01cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">sa-learn --dbpath /var/spool/amavis/.spamassassin --mbox --spam -u spamd /tmp/Junk</span></span></span><br />
</div><span style="font-style: normal;"><br />
</span><br />
<span style="font-style: normal;">sa-learn will look at the emails and will teach Spamassassin that the emails are spam. However, Spamassassin needs to be told to reload its bayesian knowledge files in order to gain this new-found knowledge.</span><br />
<b><span style="font-style: normal;"><br />
</span></b><br />
<b><span style="font-style: normal;">Tip 2</span></b><span style="font-style: normal;">: After running sa-learn, issue a kill -HUP to the spamd parent process to force a reload of the bayesian knowledge base. E.g.:</span><br />
<div style="margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">kill -HUP `cat /var/run/spamd.pid`</span></span></span><br />
</div><span style="font-style: normal;"><br />
</span><br />
<span style="font-style: normal;">In very active system the spam flies in quickly filling the Junk file. This can slow down the sa-learn processing dramatically so a good idea is to clear it down. A common way in Linux to truncate a file is to issue a command such as:</span><br />
<div style="margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
> /tmp/Junk</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
</div><span style="font-style: normal;">However, for some IMAP servers, this can produce some nasty lockups in client email software when the mail user tries to add spam emails to the folder.</span><br />
<b><span style="font-style: normal;"><br />
</span></b><br />
<b><span style="font-style: normal;">Tip 3</span></b><span style="font-style: normal;">: Clear down the Junk file(s) in an IMAP-friendly way. This means moving the file somewhere else for processing and recreating the user file rather than truncating it (note that we mv and recreate first before running sa-learn to ensure that the IMAP “folder” has only disappeared for a fraction of a second rather than waiting for a potentially very long sa-learn run to finish before recreating the file):</span><br />
<div style="margin-bottom: 0cm; margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">mv/home/username/mail/Junk /tmp/Junk</span></span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">touch /home/username/mail/Junk</span></span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">chown brad /home/username/mail/Junk</span></span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">chmod 700 /home/username/mail/Junk</span></span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 1.03cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">sa-learn --dbpath /var/spool/amavis/.spamassassin --mbox --spam -u spamd<br />
/tmp/Junk</span></span></span><br />
</div><br />
<br />
<br />
Spammers use automated tools to harvest email addresses. Publishing an email address online is a magnet for spam. This can be to your advantage if you want Spamassassin to learn about new spam messages before they arrive at your legitimate email addresses. The trick is to make spammers send <span style="font-style: normal;">spam to </span><i>honeypot</i><span style="font-style: normal;"> email</span><br />
<span style="font-style: normal;"> addresses first:</span><br />
<span style="font-style: normal;"><b><br />
</b></span><br />
<span style="font-style: normal;"><b>Tip 4</b></span><span style="font-style: normal;">: Create </span><i>honeypot</i><span style="font-style: normal;"> email addresses that route all email received at those addresses into a spam email file. This can then be used to teach Spamassassin about new forms of spam before the spammers send to your legitimate email addresses. Seed the spam email addresses on the Internet. Put them into web pages where email address harvesting software will find them but ensure that humans will not send legitimate email to them by putting up suitable messages around the email addresses.</span><br />
<span style="font-style: normal;"><br />
</span><br />
<span style="font-style: normal;">Of course, you want Spamassassin to learn about spam automatically. This means that you will want sa-learn to run periodically. </span><br />
<span style="font-style: normal;"><b><br />
</b></span><br />
<span style="font-style: normal;"><b>Tip 5</b></span><span style="font-style: normal;">: Create a cron job to run sa-learn periodically, letting it learn what is spam from the honeypot email addresses as well as the Junk folders maintained by your email users. To do this, you need a suitable cron script. Below is a template for you to use. You will need to adjust the paths to the executables and files applicable on your system. In the example below, we have called the file where the emails from the honeypots are stored </span><i>honeypot</i><span style="font-style: normal;"> which we store in /var/spool/mail. </span> <br />
<span style="font-style: normal;"><br />
</span><br />
We have assumed that users move <span style="font-style: normal;">spam that they receive into (an IMAP) file on the server called </span><i>Junk</i><span style="font-style: normal;">. In the example we show two techniques for processing this Junk user <span style="font-style: normal;">file. For </span><i>username</i><span style="font-style: normal;"> we truncate the file in an IMAP friendly manner by moving it and recreating the user file before sa-learn processes the moved file.</span></span><br />
<span style="font-style: normal;"> For </span><i>usernameX</i><span style="font-style: normal;"> we don't truncate the file. This means that the file will continue to grow in size until it's truncated by some other means. Sa-learn will ignore spam emails that it has already learned about so it is safe to not truncate a file provided that it doesn't grow to a point that sa-learn takes a long time to process it. If in doubt, truncate. </span> <br />
<br />
<span style="font-style: normal;">Also in the example below, we show how sa-learn can simply take a list of filenames on the command line which is handy if you have more than one file building up a store of spam emails:</span><br />
<div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">#!/bin/bash</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><br />
<br />
<br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">/bin/mv /home/</span><span style="font-style: normal;">username</span><span style="font-style: normal;">/mail/Junk /tmp/Junk</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">/bin/touch /home/</span><span style="font-style: normal;">username</span><span style="font-style: normal;">/mail/Junk</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">/bin/chown brad /home/</span><span style="font-style: normal;">username</span><span style="font-style: normal;">/mail/Junk</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
</span></span></span><br />
<span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">/bin/chmod 700 /home/</span><span style="font-style: normal;">username</span><span style="font-style: normal;">/mail/Junk</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><br />
<br />
<br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">/usr/bin/sa-learn --dbpath /var/spool/amavis/.spamassassin --mbox --spam -u spamd /tmp/Junk /home/</span><span style="font-style: normal;">usernameX</span><span style="font-style: normal;">/mail/Junk /var/spool/mail/honeypot >/tmp/sa-learn.log 2>&1</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><br />
<br />
<br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">#<br />
Truncate the honeypot file</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;"><br />
> /var/spool/mail/honeypot</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><br />
<br />
<br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">rm -f /tmp/Junk</span></span></span><br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><br />
<br />
<br />
</div><div style="line-height: 100%; margin-bottom: 0cm; margin-left: 1.06cm;"><span style="font-family: 'Courier New', monospace;"><span style="font-size: x-small;"><span style="font-style: normal;">/bin/kill -HUP `/bin/cat /var/run/spamd.pid`</span></span></span><br />
</div><br />
<br />
<br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-32195138728376014432010-01-19T21:48:00.000+00:002010-01-19T21:48:04.054+00:00xen "unpack list of wrong size" errorOne of the fantastic features about Xen is that when you build ane w Xen virtual machine (VM), you can specify a file on domain0 as a physical device to the VM (domU). Here is an example from a Xen<br />
machine configuration file (typically found in /etc/xen):<br />
<div style="margin-left: 0.95cm;"><span style="font-size: x-small;"><span style="font-family: 'Courier New';">disk = ['file:/xen_files/215_main_disk.img,hda1,w','file:/xen_files/215_swap.img,hda2,w']</span></span><br />
</div><div style="margin-left: 0.95cm;"><span style="font-size: x-small;"><span style="font-family: 'Courier New';"><br />
</span></span><br />
</div>The above example shows a correctly configured definition for device hda1 (which is mounted from the file /xen_files/215_main_disk.img) and hda2 (which is mounted from the file /xen_files/215_swap.img).<br />
<div style="font-style: normal;"><br />
</div><div style="font-style: normal;">However, a frequent newbie error is to forget the “file:” tag. If you entered this, for example:<br />
</div><div style="font-style: normal;"><span style="font-size: x-small;"><span style="font-family: 'Courier New';">disk = ['file:/xen_files/215_main_disk.img,hda1,w','/xen_files/215_swap.img,hda2,w']</span></span><br />
</div><div style="font-style: normal;"><br />
</div><div style="font-style: normal;">then when you run <i>xm create </i><span style="font-style: normal;">for that machine, it will output the error “unpack list of wrong size” which isn't very helpful in telling you that you forgot the “file:” tag!</span><br />
</div><br />
Please remember to link to this page if you found this useful so that others can find it too!<br />
<br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-85205389997077433292010-01-19T21:40:00.000+00:002010-01-19T21:40:49.370+00:00Disabling password expiry for specific accounts in msecThese notes are written specifically for Mandrake 10.1, however they can apply equally well to many other releases and distributions that use the msec security package.<br />
<br />
The msec package is a powerful tool for establishing tight security controls on your linux machine. It is highly customisable and comes with six pre-defined security settings that can be further customised to your requirements. However, there is a catch. The most useful setting is the <i>higher</i> level of security. With this level, though, comes a vicious password expiry regime that includes the root password. Worse still, there is a bug that sets password expiry to be immediate under certain conditions. This affects all user account in addition to the root account.<br />
<br />
The result is that your computer can be locked out to all users needing a reboot into stand-alone mode (failsafe) in order to unlock it. Not exactly the best scenario especially if your machine happens to be a server in a remote location!<br />
<br />
There is a solution to this problem though. The file <span style="font-size: x-small;"><span style="font-family: 'Courier New';">/etc/security/msec/level.local</span></span> allows you to fine tune the security settings in the msec package. You can add<br />
<div style="margin-left: 0.95cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;"><br />
</span></span><br />
</div><div style="margin-left: 0.95cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">no_password_aging_for('root')</span></span><br />
</div><br />
to <span style="font-size: x-small;"><span style="font-family: 'Courier New';">/etc/security/msec/level.local</span></span> to disable password expiry. In fact, you can call this multiple times<br />
to add any number of accounts, so for example<br />
<div style="margin-left: 0.95cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;"><br />
</span></span><br />
</div><div style="margin-left: 0.95cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">no_password_aging_for('sales')</span></span><br />
</div><br />
will disable password expiry for the <i>sales</i> login. However, there is another gotcha. The chances are that if you found this page you already have a problem with password expiry. Setting the above will not unset an expiry that is permanently expiring an account. For that you need to log into the machine and su – to root. Then you meet your new best friend, the <i>chage</i> command. This changes the password aging setting for an existing entry. So, to make <i>sales</i> never expire, you simply run:<br />
<div style="margin-left: 0.95cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">chage -M 99999 'sales'</span></span><br />
</div><div style="font-style: normal;"><br />
</div><div style="font-style: normal;">This sets sales to expire in 99999 days' time. And with the <span style="font-size: x-small;"><span style="font-family: 'Courier New';">no_password_aging_for('sales')</span></span> setting above, this will not be reset next time <i>msec</i> runs.<br />
</div><div style="font-style: normal;"><br />
</div><div style="font-style: normal;">Of course, you need to take careful note of which accounts you turn off password expiry and ensure that these passwords are changed at regular intervals when it suits you, otherwise you may be compromising the security on your machine, especially if it is online.<br />
</div><br />
<br />
<br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-79542577302681877822010-01-19T21:32:00.000+00:002010-01-19T21:32:49.952+00:00Preserving postgres default values on tables that have views and update rulesThe Postgres database has many strengths, one of the most powerful being the rules and triggers system. Combined with views, rules and triggers allow you to control access to data in underlying tables, stricting users to seeing only the data that they are allowed to see and to enforce business logic. Even complex views with data that comes from many tables through complex joins can be made updatable (insert, update and delete) through using update rules. This can dramatically simplify and speed up application development and makes<br />
rapid application development (for example using Borland's <a href="http://www.borland.com/jbuilder/index.html">Jbuilder</a>).<br />
<br />
To enable views to be used for updating the underlying datasets, you have to create update rules.This implies creating one or more rule for each update action: <i>Update</i>, <i>delete</i> and <i>insert</i>.<br />
<br />
The postgres manual for rule creation is <a href="http://www.postgresql.org/docs/manuals">here </a>(<a href="http://www.postgresql.org/docs/manuals">http://www.postgresql.org/docs/manuals</a>).<br />
<br />
Here is an example of creating a table, a view and an updatable rule, in this case for inserts:<br />
<div style="margin-left: 0.71cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">create table test1 (id serial, col1 integer not null default 10, col2 text not null);</span></span><br />
</div><div style="margin-left: 0.71cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">create view test1v as select * from test1;</span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.71cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">create rule testins as on insert to test1v do instead (</span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.71cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">insert into test1 (col1,col2) values (NEW.col1,NEW.col2);</span></span><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.71cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">);</span></span><br />
</div><br />
<br />
<br />
<br />
However, the default values for col1 and the id columns in test1 will not be preserved on insert into view est1v. Inserting a null value into these columns will cause a not-null violation. Postgres does not propogate the rules and triggers in the object beneath the view into the update rules on the view. To do this you need to explicitly add these constraints to the view using <i>alter table</i>.<br />
<br />
<div style="font-style: normal;">In this case we have the col1 default value constraint to apply:<br />
</div><div style="font-style: normal; margin-left: 0.74cm;"><span style="font-family: 'Courier New';"><span style="font-size: x-small;">alter table test1v alter column col1 set default 10;</span></span><br />
</div><div style="font-style: normal;"><br />
</div><div style="font-style: normal;">We now have the default values added to the view. In this way you can build up very complex views, abstracting a good underlying database design, adding strong security and maintaining the sort of database interface that RAD tools such as JBuilder and Delphi excel at using.<br />
</div><br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com2tag:blogger.com,1999:blog-6390279852945612246.post-63163562793664225752010-01-19T21:25:00.000+00:002010-01-19T21:25:30.369+00:00Enabling ping, NFS and ssh in Mandrake at server-grade security levelsYou have installed <a href="http://www.linux-mandrake.com/" target="_blank">Mandrake Linux</a> and have discovered that you cannot ping the machine or ssh onto it.<br />
<br />
To <b>enable pings</b>, do this:<br />
<div style="margin-bottom: 0cm; margin-left: 1.06cm;">Add/Edit /etc/security/msec/level.local<br />
add the line: accept_icmp_echo(yes)<br />
<br />
Edit /etc/sysctl.conf<br />
<br />
change the line:<br />
net.ipv4.icmp_echo_ignore_all=1<br />
to<br />
net.ipv4.icmp_echo_ignore_all=0<br />
<br />
and then run sysctl -p <br />
<br />
</div><br />
To <b>enable ssh</b>, ensure that you have ssh installed (urpmi ssh). Mandrake does not automatically enable ssh at<br />
server-grade security levels. The key here is the /etc/hosts.allow file. Ensure that you have this line in /etc/hosts.allow:<br />
<div align="LEFT" style="margin-bottom: 0cm; margin-left: 1.06cm;">sshd : ALL<br />
</div><div align="LEFT" style="margin-left: 1.06cm;"><br />
<br />
<br />
<br />
</div><div align="LEFT">There is a similar problem if you run NFS mounts on your machine. Your portmap is disabled by default at certain security levels. The key here is to enable NFS ONLY for those IPs that need access to that machine. Here is an example of enabling portmap for a subnet and also the server itself (LOCAL) within the /etc/hosts.allow file:<br />
</div><div align="LEFT" style="margin-bottom: 0cm; margin-left: 1.06cm;">portmap : 111.222.333.444/255.255.255.0, LOCAL<br />
</div><div align="LEFT" style="margin-bottom: 0cm; margin-left: 1.06cm;"><br />
<br />
<br />
</div>Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-68499618606594494372010-01-19T21:04:00.000+00:002010-01-19T21:04:09.339+00:00How do I search for keywords in OpenOffice word document files?This is the tool that you need:<br />
<a href="http://www.danielnaber.de/loook/">Loook</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-7990300856134534902010-01-19T18:27:00.000+00:002010-01-19T18:27:54.624+00:00Compiling PHP versions >= 4.3.0 failsYou downloaded the latest PHP version with its built in GD library code and tried to compile it. All goes well until it fails with this nto so meaningful message:<br />
<br />
<br />
<pre style="margin-left: 2cm;"><i>In file included from gdft.c:37:</i>
<i>/usr/local/include/freetype2/freetype/ftglyph.h:104: <b><span style="color: black;"><span style="background-attachment: initial; background-clip: initial; background-color: #a0ffff; background-image: initial; background-origin: initial;">parse </span><span style="background-attachment: initial; background-clip: initial; background-color: #99ff99; background-image: initial; background-origin: initial;">error</span></span></b></i>
<i>before `<span style="background-attachment: initial; background-clip: initial; background-color: #ffff66; background-image: initial; background-origin: initial;"><b><span style="color: black;">FT_Library</span></b></span>'</i></pre><div style="margin-left: 2cm;"><br />
<br />
</div><div style="font-weight: medium;"><span style="font-family: 'Times New Roman';"><span style="font-size: small;">You consider crying, but then you remember that the Kieser.net guys know their stuff and maybe they have the answer? Well, you are in luck. This error normally means that your /usr/local/include directory has an old <i>freetype</i> subdirectory as well as the new <i>freetype2</i> directory that contains both freetype 1 and 2 include files in it. The solution is simple:</span></span><br />
</div><div style="font-weight: medium;"><span style="font-family: 'Times New Roman';"><span style="font-size: small;"><i>mv /usr/local/include/freetype /usr/local/include/old.freetype</i></span></span><br />
</div><div style="font-style: normal; font-weight: medium;"><span style="font-family: 'Times New Roman';"><span style="font-size: small;">and try again! It should work now! ;-)</span></span><br />
</div><br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-22587170781885964082010-01-19T13:46:00.000+00:002010-01-19T13:46:41.299+00:00NFS mounts that hang with status "D"<b>Dang!</b><br />
<b><br />
</b><br />
You got a great Linux office going or maybe a network of servers working together. They share their drives and data through NFS mounts.<br />
<i><br />
</i><br />
<i>Then disaster happens</i>! A NFS server goes down and every machine locks up! The machines trying to mount the NFS partition hang. The processes trying to access the NFS disk (probably your root partition) freeze and won't die!<br />
<br />
The reason this happens is simple: A disastrous decision taken by Sun who developed NFS in the first place, compounded with distributions that simply haven't figured it out yet, means that NFS partitions are mounted, as a default, with the <i>hard</i> option. This tells the computer that it cannot function without that NFS mount ... so it hangs, trying and retrying forever to get that mount up and running.<br />
<br />
Great for diskless work stations.<br />
<br />
Horribly stupid for the real world!<br />
<br />
Fortunately the solution is as easy as drinking a cappucino. Simply add the option <i>soft</i> into the /etc/fstab entry for the NFS mount. This tells NFS to try but not hang up. Adding <i>bg </i>into the options tells it to background the retries, meaning that reboots and mounts will keep the retries in the background allowing the machine to continue processing as normal.<br />
<div style="margin-left: 0.74cm;"><br />
</div><div style="margin-left: 0.74cm;">Here is an example NFS line with<br />
suitable options set:<br />
<br />
<b>sample.com:/kieser/is/great_dir <br />
/sms_team nfs <br />
soft,bg,intr,timeo=10,retrans=2,retry=2,user,owner,exec,dev,suid,rw 0 0</b><br />
</div>Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-81462355885872866812010-01-19T11:02:00.000+00:002010-01-19T11:02:44.242+00:00Unicode, PostgreSQL, JDBC and truncated data<span class="Apple-style-span" style="font-family: Arial, sans-serif; font-size: large;"><u><b>The problem</b></u></span><br />
<br />
<b>Unicode</b><br />
Unicode is a 16 bit character encoding that is destined to replace ASCII as the universal character encoding standard. Unlike ASCII, which was design for (and is largely limited to) the american character set, Unicode caters for the entire world's character sets, even the awesome array of Chinese characters! If you can type it on a keyboard, you can store it in Unicode. At last! A truly global, properly usable character encoding that doesn't assume that the entire world is an additional state of the USA!<br />
<br />
Er, right.<br />
<br />
Problem number one is that ASCII is a <i>de facto</i> standard across most of the world's machines and software.and things get seriously tricky when you start trying to map local incarnations of character sets that have been shoe-horned into the basic ASCII encoding (limited to 7 or 8 bits) to the universal big brother of<br />
them all: Unicode.<br />
<br />
<b>PostgreSQL</b><br />
<br />
<a href="http://www.postgres.org/">PostgreSQL</a> is the best database around. Yes, we are biased. Yes there are other excellent databases (you may think Oracle or Ingres, but we think of <a href="http://www.mysql.com/">MySQL</a>, which very seriously rocks as well. PostgreSQL supprts many different types of character sets (see the <i>create database</i> command for more details of its <i>encoding</i> option).<br />
<br />
<div style="font-style: normal;"><b>Symptom</b><br />
</div><div style="font-style: normal;"><b><br />
</b><br />
</div><div style="font-style: normal;">The problem is that when you connect to PostgreSQL via JDBC and you select text rows as a string from a<br />
table, if those rows contain 8 bit characters (for example a pound (£) sign for the UK), then you may find that the data for that column gets truncated just before that character. I.e., this command fails:<br />
</div><div style="font-style: normal;"> read_rs.getString(1);<br />
</div><br />
Trawling the news groups, it seems that this is caused by Java, which also uses Unicode internally, not being able to map the character coming back from Postgres into a valid Unicode character. Often, the problem lies with the original insert into the Postgres table. The character in question is sent from some client software (with its own character set) to Postgres (which stores it in the character set for that database). On insert, Postgres doesn't check that the value stored in the table is a legal map from the client character encoding set, it merely stores that ASCII value for that character (there is a mapping option that you can turn on when you<br />
build Postgres, see the <a href="http://www.postgresql.org/idocs/">Postgres Manual</a> for more information on this).<br />
<div style="font-style: normal;"><br />
</div><div style="font-style: normal;">It appears that the data is truncated due to a fault in the error handling of the JDBC software.<br />
</div><div style="font-style: normal;"><br />
</div><div style="font-style: normal;">We don't really agree with this analysis. Although the common concensus is that this is a Postgres fault, we're not actually convinced about this. There are numerous reports of the same problem with different databases, including Oracle, MySQL and DB2. We think that the problem lies with the JDBC system and it not being able to determine what the character set is that the data is stored in. This may well come down to the database<br />
supplying more information to JDBC but it may equally be that JDBC needs to examine the environment or use some other resolution mechanism<br />
</div><div style="font-style: normal;">.<br />
</div><div style="font-style: normal;"><span style="font-family: Arial, sans-serif;"><span style="font-size: medium;"><u><b>The solution</b></u></span></span><br />
</div><br />
<div style="font-style: normal;">The best solution is to force Java to read the column as a set of byte values and then explicitly tell Java what the character set is that the table is stored in and then it can do the translation no problem! SO, converting this line:<br />
</div><div style="font-style: normal;"> read_rs.getString(1)<br />
</div><div style="font-style: normal;">into this line:<br />
</div><div style="font-style: normal;"> new String(read_rs.getBytes(1),"ISO-8859-1")<br />
</div>does the trick. Of course, in the above example the table was in ISO-8859-1 format. This is the most likely format if Unicode translations are failing, but you do need to check which character set is used in you local software! <a href="http://czyborra.com/charsets/iso8859.html">See</a> <a href="http://czyborra.com/charsets/iso8859.html">here</a> for a list of possible character codes, what they are, and a very handy discussion of the codes and the characters in each coding.<br />
<br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-57735361948653249162010-01-19T10:58:00.000+00:002010-01-19T10:58:34.283+00:00Cron job to check for RAID disk failureLinux's software RAID handling is fantastic, but how do you <i>know <span class="Apple-style-span" style="font-style: normal;">if one of the disks have failed? RAID is designed to not have a single point of failure which means that if one of your disks goes West, you won't know about it. In a busy server environment you probably don't have the time to keep checking your Linux kit. Linux has a habit of running reliably for years and years until the hardware fails or you need to upgrade the system.</span></i><br />
<i><span class="Apple-style-span" style="font-style: normal;"><br />
</span></i><br />
<div style="font-style: normal;">Well, fear not! Kieser.net to the rescue! We have this neat little script that does and elementary check for disk failure and then emails you if it detects a failure. You should install it on your server as root, and <i>chmod 500</i> so that it is executable by cron. Of course, you also need to use <i>crontab -e </i>to make cron run it at a sensible frequency.<br />
</div><div style="font-style: normal;"><br />
</div><div style="font-style: normal;">Here is the script for you to cut and paste into a suitable file:<br />
</div><br />
<br />
<div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>#!/bin/bash</b><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.81cm;"> <br />
<br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>LOG_FILE=/tmp/raid_check_$$</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>SYSTEM=`uname --nodename`</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>MAILTO='root@kieser.net'</b><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.81cm;"><br />
<br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>echo "The $SYSTEM system has RAID failures on it." >>$LOG_FILE</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>echo "Below is the output from /proc/mdstat" >> $LOG_FILE</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>echo "===========================================" >> $LOG_FILE</b><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.81cm;"><br />
<br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>cat /proc/mdstat | egrep 'md.*raid' | fgrep -i '(f)' >> $LOG_FILE</b><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.81cm;"><br />
<br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>if [ $? -eq 0 ]</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>then</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>cat /proc/mdstat >> $LOG_FILE</b><br />
</div><br />
<div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>echo "===========================================" >> $LOG_FILE</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>mail -s 'URGENT: RAID disk failure detected' $MAILTO < $LOG_FILE</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>fi</b><br />
</div><div style="margin-bottom: 0cm; margin-left: 0.81cm;"><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>rm -f $LOG_FILE</b><br />
</div><div style="font-style: normal; margin-bottom: 0cm; margin-left: 0.81cm;"><b>exit 0</b><br />
</div><br />
<br />
<br />
<br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.net</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0tag:blogger.com,1999:blog-6390279852945612246.post-23248747554290357802010-01-19T10:53:00.000+00:002010-01-19T10:53:56.406+00:00Recovering a RAID disk back into a RAID deviceOkay, so you have been clever! You figured that with Linux you can build a RAID using nice cheap IDE disks. Linux's fantastic software RAID feature allows you to do this saving loads of money on harware<br />
RAID and expensive SCSI disks. Maybe you did the easy thing and used a distribution like <a href="http://www.linux-mandrake.com/" target="_blank">Mandrake Linux</a> that makes it oh so easy to set up.<br />
<br />
<i>Then disaster happened</i>! Maybe you did a forced reboot, maybe something else happened, but when the reboot had finished you did <br />
<br />
<div style="margin-left: 0.56cm;"><b>dmesg | less</b> <br />
<br />
</div>and you saw something like this in the log:<br />
<div style="margin-left: 0.58cm;"><b>hdf7's event counter: 00000006</b><br />
</div><div style="margin-left: 0.58cm;"><b>hde5's event counter: 00000003</b><br />
</div><div style="margin-left: 0.58cm;"><b>md: superblock update time<br />
inconsistency -- using the most recent one <span class="Apple-style-span" style="font-weight: normal;"><b>freshest: hdf7</b></span></b><br />
</div><div style="margin-left: 0.58cm;"><b>md: kicking non-fresh hde5 from array!</b><br />
</div><div style="margin-left: 0.58cm;"><b>unbind<hde5,1></b><br />
</div><div style="margin-left: 0.58cm;"><b>export_rdev(hde5)</b><br />
</div><br />
Oh boy! Quick as a flash you look into the status of the array:<br />
<div style="margin-left: 0.58cm;"><b>cat /proc/mdstat</b><br />
</div><br />
and it looks bad:<br />
<div style="margin-left: 0.64cm;"><b># cat /proc/mdstat</b><br />
</div><div style="margin-left: 0.64cm;"><b>Personalities : [raid0] [raid1]</b><br />
</div><div style="margin-left: 0.64cm;"><b>read_ahead 1024 sectors</b><br />
</div><br />
<div style="margin-left: 0.64cm;"><b>md2 : active raid1 hdf7[1]</b><br />
</div><div style="margin-left: 0.64cm;"><b>39262720 blocks [2/1] [_U]</b><br />
</div><div style="margin-left: 0.64cm;"><b>md1 : active raid0 hde2[0] hdf6[1]</b><br />
</div><div style="margin-left: 0.69cm;"><b>497792 blocks 64k chunks</b><br />
</div><div style="margin-left: 0.69cm;"><b>md0 : active raid1 hde1[0] hdf5[1]</b><br />
</div><div style="margin-left: 0.69cm;"><b>505920 blocks [2/2] [UU]</b><br />
</div><div style="margin-left: 0.69cm;"><br />
<br />
<br />
<br />
</div>Now, in the above, /dev/md2 is the root partition on your machine (of course this is only an example and it may NOT be this device but some other /dev/md* device). It s<i>hould</i> be a RAID level 1 (mirrored) but there is now only one disk in that array!<br />
<br />
What to do?<br />
<br />
Well, you need to restate the kicked out disk (in this case, /dev/hde5). There is a useful command to do this:<br />
<div style="margin-left: 0.71cm;"><b>raidhotadd /dev/md2 /dev/hde5</b><br />
</div><div style="font-weight: medium; margin-left: 0.71cm;"><i>(</i><b>NOTE: </b><i>you need need substitute your own correct devices. The above is an example only)</i><br />
</div><br />
That will rebuild the dirty mirror disk from the main mirror disk. It will bring the RAID back to a fully flying 2-disk mirrored setup provided, of course, that the disk doesn't have a fault making it fail. While the rebuild is happening, you can monitor the rebuild by:<br />
<div style="margin-left: 0.76cm;"><b>cat /proc/mdstat</b><br />
</div><div style="margin-left: 0.76cm;"><b><br />
</b><br />
</div>It <b>may</b> be that your disk fails to join the araay and after raidhotadd completes, you see something like this:<br />
<br />
<div style="margin-left: 0.74cm;"><b># cat /proc/mdstat</b><br />
</div><div style="margin-left: 0.74cm;"><b>Personalities : [raid0] [raid1]</b><br />
</div><div style="margin-left: 0.74cm;"><b>read_ahead 1024 sectors</b><br />
</div><div style="margin-left: 0.74cm;"><b>md2 : active raid1 hde5[0](F) hdf7[1]</b><br />
</div><div style="margin-left: 0.74cm;"><b>39262720 blocks [2/1] [_U]</b><br />
</div>Note the (F) which means that the disk failed. Now hard drives are extremely reliable and it us unlikely that your disk is toasted (although you can always assume this to be safe). There is a great Linux command, <i>badblocks</i> that will scan your disk and mark off the bad blcoks on it. You can then safely add it back into the array. Please note though:<br />
<div style="font-style: normal; margin-left: 0.74cm;"><span style="color: #ff3333;">Only run this on unmounted disks</span><br />
</div><br />
<div style="font-style: normal; margin-left: 0.74cm;"><span style="color: #ff3333;">It takes a LONG time to run.</span><br />
</div><br />
Simply run:<br />
<div style="margin-left: 0.79cm;"><b>badblocks -f /dev/hd*</b><br />
</div>where /dev/hd* is the device name for your drive. In the <i>example </i>above this would be /dev/hde5. After the badblocks has run, try to raidhotadd the disk back into the array again.<br />
<br />
You have to admit it: Linux is HOT!<br />
<br />
Honeypot: <a href="mailto:spam@kieser.net">spam@kieser.ne</a><a href="mailto:spam@kieser.net">t</a>Bradley Kieserhttp://www.blogger.com/profile/10723853755510927711noreply@blogger.com0