Louwrentius - Uncategorized

I think you might be fooling yourself with AI

2026-07-23T12:00:00+02:00

If you are enthusiastic about AI (LLMs), you may be fooling yourself. Do not consider this an insult, but a kind-hearted warning.

Remember what Richard Feynman said:

The first principle is that you must not fool yourself
and you are the easiest person to fool.

source (backup)

He made this statement as part of the Caltech's 1974 commencement address. In this speech he strongly advocated for proper scientific integrity.

I feel it can be difficult to maintain proper 'scientific integrity' on a personal level as AI is a fast-moving target. The tools feel unreasonably capable, using an LLM makes you feel more productive when writing code for instance.

Yet as of today, there is no independent scientific evidence¹ (that I'm aware of) that people or organizations are indeed more productive² when using AI.

Are you really more 'productive'?

Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.

Using AI may feel faster and more productive, but you might be fooling yourself. You can never go back in time and measure yourself doing the task without using AI tools and compare the results.

There is a reason why (scientific) experiments are often blind or double-blind. We want prevent personal bias from influencing the outcomes skewing the results and making us believe things that aren't true. Not so easy to do on your own.

This is why I'm not really swayed by personal testimonies and anecdotes about how AI makes people more productive. How they vibe coded some app they would otherwise not have built³.

Do not be fooled by the mass layoffs 'due to AI' proving that AI is making people more productive. Sam Altman himself called many of these layoffs 'AI washing'. These are regular layoffs after a period of over-hiring and now the market corrects itself. Because aside from AI-related businesses, it seems that the economy isn't doing great.

Furthermore, some of these layoffs seem to be specifically to free up money to buy AI resources. Management by fear of being left behind. Firing people to get money to pay for unproven technology that may never be capable to replace the people being fired.

AI is unsustainable

Even if you still think AI is really making you more 'productive', enjoy it while it lasts.

According to this article ChatGPT starts losing money when you use your $200 subscription abouve 11%. If you'd use the full 100%, that same $200 subscription would allow you to use $14,000 worth in API pricing. That seems crazy to me.

Whatever you think your productivity might be due to AI, I don't think it's worth the 'real' cost once the vendors stop heavily subsidizing AI usage. I really wonder how much usage AI would see if it would be just priced at cost. Would you feel as productive if the price would not be $200 but $2000 a month⁴? A $200 subscription is not that big of a gamble, but any higher and it may feel less and less worth it.

Disclosure: my own bias against AI

Now it's time to be really open about my own bias. I'm an AI sceptic. I'm quite influenced by Ed Zitron's reporting. And I'm also standing by my own idea that AI will run out of money and just be turned off due to the huge operational cost.

It might feel unfathomable, unreasonable that this can happen. Maybe Google, AWS or Microsoft scoops up OpenAI or Antropic when they reach the end of the runway. But the economics won't change. Are they going to subsidize AI indefinitely?, to what end?

I'm open to suggestions how this is going to play out. Close to a trillion dollars invested in AI. Where will the money come from to recuperate all these investments? If we disregard the AI (adjacent) companies, the USA economy seems to look bleak (August 2025).

I myself can't just close my eyes for all the externalities of AI. The energy usage in the context of climate change. The data centers disrupting poor communities. The large-scale theft of intellectual or copyrighted property. The enshitification and the enslopification of the internet. The mass hoarding of memory causing inflation.

I'm not convinced the bad of AI is - or can be - compensated by the perceived good AI does. It's not possible to separate the tool itself from it's externalities. This is why I'm not using AI. I'm not using AI on moral, ethical grounds.

Hacker News discussion: here

Post Script

I found the comments on hacker news quite revealing. Most are very dismissive without engaging at all with the actual point of this article.

It is true that the cited Metr study was followed up with a more recent study from 2026 that showed more productivity when using AI. The authors are also pointing out they paid way less and they had a problem where developers didn't want to code without AI.

What was actually interesting about the initial study is the discrepancy between what developers thought and what they measured in terms of productivity. They were measurably fooling themselves.

Furthermore, it seems that people are quick to respond and not actually read and comprehend what I wrote. There are two key points I'm making:

Your perception of productivity may be off / you may be fooling yourself
In addition, I question and you may fool yourself thinking that the AI tool is worth the unsubsidized cost in relation to the perceived productivity gain.

At least I will try to be more explicit in my writing to get this point across.

I must say I'm amazed at how fast people dismiss this blog post and make claims about AI productivity, without anyone providing anything else but personal anekdotes.

This may be due to my confirmation bias that I was unable to find any. ↩
Whatever productive really means in this context. ↩
how much value it delivers or how much revenue it creates is often not discussed. And don't fall for suvivorship bias. ↩
Or would you get stressed out and feel pressure to use the AI more to justify the expense and burn out? ↩

The AI mirage or why I think the hype can't sustain itself

2026-06-30T12:00:00+02:00

If you zoom out enough, everything is a black box where we don't know about the inner workings, but we can learn about the black box by putting stuff in and observing the output.

Let's pretend that a large language model or LLM is such a black box. If we observe a LLM, we learn - amongst other things - that the output is 'correct' 99%¹ of the time.

This is a very fundamental and important observation. Computers are known for their correctness, for their reliability. We know that if we run a function with input A, we get output X, no matter what. Yes, packets can be lost, memory can go corrupt but those things go wrong in a predictable way. We check and retransmit packets, we use ECC memory. And data is formatted in such a way that we can detect lost packets or corrupted memory in the first place. Our entire world relies on it.

Imagine that a system can't be trusted. That 1+1 isn't always 2. Only 99% of the time. How valuable can such a system be? That probably depends on the circumstances, but we know one thing for sure: we can't trust the output, it must be checked. It doesn't matter how, but it must be checked for correctness and that requires a person.

We see it with self-driving cars. It's really impressive what is possible. But they aren't true self-driving². A person needs to be sitting at the wheel, keeping their attention focussed on traffic as if they would be driving themselves, so they can intervene when the AI makes an inevitable mistake.

Although this post isn't about self-driving cars, we know that humans have a tenancy to get distracted and bored if we aren't actively engaged. We can argue that either we people keep driving ourselves, or we need 100% reliability, so we can remove the steering wheel and read a book while the car drives itself. Only 100% is good-enough, 99% doesn't cut it. What did we actually solve if I'm still required to 'drive' the car because of the 1% chance things go wrong?

In the case of an LLM, how much time is saved by letting a person check the output of an LLM? Can that saved time justify the actual operating cost of the LLM, as opposed to the subsidized cost AI vendors are currently charging?

In case of writing code, an LLM can probably create more functionality and features in a week a team of 100 engineers can validate in a year. So no matter what how fast the LLMs really are, the people are the bottleneck we can't circumvent.

That is, if we care about correctness, about quality, about stability and so on. But if we don't, why do something in the first place, regardless of an LLM being involved or not?

So this is why I think it's impossible for the AI hype to come through on the sky high promises their absurd valuations suggest. I'm not saying that AI may not be valuable³ but probably orders of magnitude less valuable than people want us to believe.

I admit, this isn't an original idea, people have voiced this idea in different, maybe more succinct ways. Yet I think it is worth repeating.

Update - but humans aren't accurate either!

Why replace people with something objectively worse and more unpredictable? Humans check human output. Now humans check LLM output. Throughput is the same, but we fired a person and the remaining person is miserable due to the nature of the work.

P.S.

I also don't understand why organizations would make LLMs a very integral part of their processes, only to discover that models are changed and tweaked, resulting in wildly unpredictable and different output.

Sometimes, when the light hits an LLM just right and you squint your eyes, it takes the shape of a crypto currency. At least that's what I see.

the number is based on nothing, in practice it's probably worse and the actual value is not important except to note that it's not 100%. ↩
don't be fooled by Waymo, it's still a limited system and operated remotely by people when needed. ↩
I'm ignoring the energy 'waste', pollution, the IP theft, the copyright infringement, the AI-induced self-harm. On and on the list goes. ↩

I build a 10 inch mini rack from aluminium extrusions

2026-06-20T12:00:00+02:00

I built a 10-inch mini rack from aluminium extrusions and I had a lot of fun doing it. I want to share my build in this post.

In January of 2025, Jeff Geerling released a video about 10-inch mini racks. I was absolutely oblivious to this new trend, and I instantly knew that I wanted to build one for myself some day, although I didn't have a real use for one.

That said, I've been working on a virtualization project recently and I've bought six 1L PC's (three for each simulated datacenter).

Because these 1L PCs are small, they have huge external power bricks. As these mini PCs are also connected to two networks, they create a huge mess on my desk. Finally a reason to build a mini rack, to tidy things up!

There are a few different brands of mini rack for sale, but I wanted to make one for myself. Prebuilt racks (kits) are not cheap for what they are so I wanted to try and see if I could build one myself for less money. Turns out you can buy 20mm aluminium extrusions and accompanying components to build your own rack.

(Aluminium extrusions have a standardized 'groove')

Aluminium extrusions are bars with a groove on all four sides. These bars have a standard format and you can slide all kinds of equipment in there and lock it in place with set screws. It seems to be used a lot for home made 3D printers, CNC machines and whatnot. In the picture above a special corner piece is used to connect three bars together, fixed in place with set screws.

The L-brackets can be used to create T-sections within a frame to sturdy the structure and provide additional mounting points. In my rack, the middle post carries the back of the shelves holding the mini PCs.

These sliding cage nuts (M5) can be used to attach anything anywhere. In this example we used four of them to hold the side panels in place. These cage nuts can also be used for their intended purpose: mount 10 inch rack-mount equipment.

In this picture below, some cage nuts are left that hold the side panel in place. Also notice in the upper left that I've used cage nuts to attach some black cable tie holders that in turn keep cables in place.

The computer trays

I planned on ordering 10 inch 1U shelves and be done with it. Unfortunately these metal shelves are too expensive for my taste and would have cost more than the aluminium frame including components (I need 8 shelves).

Many 10-inch rack builds - such as the ones featured by Jeff Geerling - use 3D-printed face plates to mount various kinds of equipment. Jeff showcased some of these models in separate videos. That said, I decided against using 3D printed shelves.

First of all, I don't have a 3D printer and as useful as 3D printing can be, I feel that 3D printers often turn plastic into landfill. I'd probably feel differently if (more) sustainable materials would have been used¹.

So instead, I chose to order cut-to-size aluminium sheets and I used the L-brackets to hold them in place.

The aluminium shelves turned out OK, but they are not ideal. The 1mm thick plates do bend slightly under the weight of the computers, although it's still fine. Aligning the four L-brackets on the same horizontal plane was a pain. Filing off the sharp corners of each plate was no fun, I should have ordered them with rounded corners.

I think these aluminium plates create an open design that is better for keeping the machines cool. There are many 10-inch self 3D models available for these 1L PCs but they all create a tight collar around the front bezel of the computer, which looks amazing, but I don't think it's great for airflow. Regular metal shelves would also have been fine.

10 inch power distribution

As you can see below, the backside shows the internals of the case are a bit of a mess². The truth is that I've should made the rack at least 1U higher to accommodate the very lengthy cabling of the power bricks. All the cabling does fit, but it's not easy to make it clean looking and also give the power bricks - which lie at the bottom - some airflow.

No price for cabling management and neatness, that's for sure

I've used two 10-inch rack mount power distribution units from Brenenstuhl. They were cheap but they unfortunately didn't fit in a horizontal position. The PDUs are the only actual 10-inch rack-mount component in the entire build and the fact that they didn't fit felt ironic.

The cause is simple: in a 19-inch or 10-inch rack, the square holes holding the cage nuts are 'flat sheets', so the power cable sticking out of the side of the PDU can flow behind those square holes. If you use the 20mm aluminum extrusions, there is a 20mm bar in the way. This is why I had to mount the PDUs vertically, which did work fine. At the top, the PDUs are kept in place with another L-bracket clamping the PDUs firm against the rail.

The external power bricks of the 1L PCs' are a huge pain. Having six cords and external adapters was absolutely no option for me, I wanted a fully self-contained rack. If you ever build your own rack, try to use computers that have a power supply build-in.

I wish I could power these 1L PCs with a 'power shelf'. In this case, a power supply with enough capacity to sustain these PCs at the required voltage and with the proper brand-correct power jack, maybe like this.

Cooling

All power sockets are in use by the six 1L PCs and the two network switches, so how are we going to power the two cooling fans³?

Fortunately there exist USB-to-fan-header cables. It feels dirty but it isn't. USB is only 5 volt but these adapters contain a boost converter that outputs 12 volt for the fans. The power draw of these fans (around 1W) is well below the threshold of 4.5W for a USB3 port.

The open backside probably doesn't help with cooling and Ideally I'd fill in the gaps. Due the the irregular shapes, I feel it's too much effort. I do feel a proper airflow at the front of the 1L PCs, so I think it's 'certified good enough⁴'.

Networking

The 1L PCs have two network connections, one to each switch. The 3Com switch at the top is 1 Gbit, the TRENDnet switch is 2.5 Gbit. I used the Conceptroning ABBY12G USB3-to-2.5Gbe adapter to connect the 1L PCs to the switch.

The 2.5 Gigabit network is a backend network for live migration of virtual machines amongst other traffic. I'm able to achieve line-speed with both iperf3 and virtual machine migrations.

The gigabit 3com switch is at least 26, maybe 27 years old!

The rack has become a bit crowded due to all this network connectivity, as seen on earlier pictures. It didn't help that I ordered short UTP cables that are quite stiff.

Panels

The side panels are sheets of anodized aluminium which look good, in my opinion. I forgot to order panels for the top and bottom. Therefore, I decided to go with some wood panels instead of aluminium sheet metal.

A local shop offers scrap wood for peanuts and also cuts it to size for a 'few peanuts more'. Very handy if you don't have the right equipment to make straight, clean cuts of wood.

Although not pictured, the bottom panel is kept in place with screws locking into cage nuts. These screws also hold the rubber feet in place.

Problems

A lack of access to the VGA/DP ports:

If I lock myself out of a machine, I need to remove one or more fans and connect a VGA/DP cable to the back of the affected machine. This also means moving a lot of UTP cables to the side. I can attach a keyboard to the front but attaching a monitor is thus a real pain. In more regular rack builds, you can add 'keystones' that extend ports to a panel on the outside of the case, where you can connect to an interface without any issue.

Cost

Item	Total
Aluminium sheets	82.95 €
Aluminium extrusions	26.22 €
Rack mount small items	79.96 €
wood	7 €
shipping costs (over all orders)	20 €
USB to 2x 4-pin fan header	12 €
10 Inch PDU 2x	43 €
Network cables	42.15 €
Total Price	313.28 €

All prices include 21% Dutch sales tax. The total price excludes the two Noctua fans, which would probably add another 40 euros.

The price is a bit inflated because I overbought small items for rack building. If I opted for wood paneling instead of aluminum panels, that would probably also cut the paneling cost in half.

How I use this mini rack

The mini PCs are running Debian and they all act as virtualization hosts, using KVM. The gigabit ports are used for the management and provisioning network (PXE+TFTP+iPXE+HTTP). The 2.5Gbit network is used for virtual machine migration and a VXLAN network that encapsulates all the different virtual machine networks.

By default, this rack is off. When I want to use it, I turn on a Zigbee power adapter and after a delay, wake-on-lan packets are sent to all six machines to power them on.

Power consumption

The two switches, two fans and six 1L PCs together use around ~90W idle.

Evaluation

I'm quite happy with this build. Cost was acceptable, I think it looks decent enough and it really cleans up my desk. Although cable management is clearly not my strong suit, I feel it's an overall improvement. Maybe a few handles for carrying would be a nice future addition, but for now this rack is finished.

Did I save any money? I'm not so sure.

Hacker News discussion

Acknowledgements

Jeff Geerling for introducing me to 10 inch racks
This build by Logan Marchione inspired me to further look into aluminium extrusions

If you feel differently about 3D printing that's OK. I'm not looking for a discussion or argument about the topic, I'm only stating my motivations. ↩
it's so, so much cleaner than the mess of wires on my desk. I'm already happy with this improvement. ↩
The fans I used are of different size and type, in part because it's what I had lying around. ↩
https://www.youtube.com/SuperfastMatt ↩

I think Rutger Bregman & The School for Moral Ambition are full of shit

2026-06-08T12:00:00+02:00

Disclaimer: I'm just a random person on the internet and this article is my personal opinion. This article was written without AI.

The first time I heard from Rutger Bregman was due to his remarks at Davos about taxing the billionaires. His speech went viral and I was rooting for him. Later on he got to troll Tucker Carlson about Carlson being a millionaire working for billionaires. Absolutely fantastic.

A few books in, Bregman started the School for Moral ambition in 2024. Although he is a co-founder, he seems to be face of this non-profit organisation, doing a lot of high-visibility interviews and he's the author of the book "Moral Ambition".

The idea behind the School for Moral Ambition (SMA) is quite simple: use your talent to address the world's biggest problems. To paraphrase: "don't become a consultant at McKinsey but do something more valuable and worthwhile with your talent". The argument is similar to what Steve Jobs said to John Sculley: "Do you want to sell sugar water for the rest of your life, or come with me and change the world?"

It sounds all very noble, but I've become a skeptic for multiple reasons.

Bregman rubbed me the wrong way in the last podcast episode of the "De Rudi & Freddie Show" in 2024. He confronted co-host and award-winning investigative journalist Jesse Frederik and openly questioned if his work really made a difference. Bregman implicated that Frederik could do other things that would be much more effective and be more impactful than his investigative journalism (I'm paraphrasing here).

I'm absolutely comfortable with questions like these, although it can feel confrontational. Yet I really felt it came from a somewhat righteous and smug moral superiority. I felt this way because I haven't seen any evidence¹ that SMA has done anything of substance that warrants all the money involved.

Bregman drank the AI cool-aid

Now I already learned that Bregman was really enamoured with AI, and I'm absolutely not, for the usual reasons. That said, fast forwarding to today, I truly believe he has lost the plot, and this recent video (June 2026) is evidence of it.

He is comparing skepticism of artificial intelligence with climate change denial or climate skepticism. He sees AI as a tool that is beneficiary, just not in the hands of billionaires but in the hands of governments (as I understand it). He believes that AI will create an utopia where people can actually work less hours as Keynes 'wrongfully' predicted

Unfortunately for Bregman, as far as I can tell, there is no (scientific) evidence that AI is actually capable of having such a huge impact. Meanwhile compare this to the mountains of evidence for human-caused climate change, it would be a different story. I would expect a different argument from a former journalist. To reiterate: the case for climate change is rooted in a vast body of scientific research. The case for AI creating a 15-hour work week seems mostly backed by wishfull thinking.

I feel that Bregman isn't really aware of his own biases, fuelled by his own personal experience using AI. He paints all the AI critics as 'luddites'² and that AI-skeptics are on the wrong side of history, just as the climate deniers are. Comparing AI skeptics to climate change skeptics feels like a cheap rhetoric trick that doesn't do justice to the criticism and skepsis surrounding AI.

Bregman also believes that stopping AI or 'shutting it down' doesn't make sense. Because if we shut AI down in our countries, other countries or regimes will build AI and get an advantage over us (he shows China as an example). The age old "if we don't, somebody else will", that has excused bad behaviour for all of eternity.

The reality is that voters and the larger public never had any say how this AI technology is deployed. That we take into account the impact on communities, people, the use of energy and other resources. We never had any say in this. The entire AI-push seems anti-social, anti-democracy.

Yes, the companies now pushing AI upon us are so huge and valued so high they are dwarfing the GDP of entire nations within Europe as Bregman suggests. Yet, I think there is strong evidence these numbers are not real. I sense such a lack of critical thinking and healthy skepticism about his own claims. He is dismissing the work of people like Ed Zitron who has unearthed a ton of evidence that AI doesn't deliver (or not nearly as much as is widely claimed) and there's a bubble that may collapse at some point. As I see it, Bregman only has a well-produced narative, but nothing more: it seems pure rhetoric.

And then I learned something about SMA that made me really disappointed.

The School for Moral Ambition took money from Bill Gates

Bregman has been quite positive about Bill Gates contributions to society, mainly pointing at his work on fighting malaria. Unfortunately, it seems that Bregman has not done his due diligence about Bill Gates and his foundation.

Fortunately, Tim Schwab - an investigative journalist - has investigated the Bill Gates foundation and the result has been documented in the book "The Bill Gates Problem: Reckoning with the Myth of the Good Billionaire". In this book, Schwab has demonstrated how problematic Bill Gates and his foundation really is. Remember that Bill Gates had deep ties to Jeffrey Epstein, which was cited by his ex-wife as a reason to file for divorce.

Tim Schwab had contact with Bregman about this and I think it doesn't look good for Bregman. It's really bad for a former journalist to cite a source as evidence for Bill Gates good work that has financial ties to Gates. Schwab repeatedly tried to contact Bregman about this, but got the statement through Bregman's agent Anne Strunk:

“He’s not a fan of your style of journalism, which seems grounded in bad faith instead of curiosity,” Strunk noted in an email.

When Schwab wrote his article in 2025, it wasn't known that Bregman and his School for Moral Ambition was funded in part by Bill Gates, Schwab discovered this by accident. (It's now disclosed on the website at the bottom).

Now for the life of me, I can't fathom boasting about moral ambition and accepting money from Bill Gates. I just can't. And they know it looks bad because if an organisation omits (at first) mentioning receiving funding from Bill Gates, you know they know how bad that looks. Lying by omission.

If SMA had any actual moral ambition, I think it would have been a cooperative. (Now it's just a typical hierarchical and opaque organisational structure with the 'owners' including Bregman, at the top.) I would expect that this organisation would also be extremely transparent where the money is spent on exactly. The impact report didn't really clarify anything for me. It's all testimonials, stories and anecdotes.

It gets so much worse

My jaw dropped when I read this article by Anita Naidu. I learned that Dutch journalist Marieke Kuypers discovered that the USA branch of SME runs out of the offices of crypto billionaire Mike Novogratz and that Novograt's sister sits on the SMA's board. Mike Novogratz donated a milion dollars to Trump.

At this point, I think that both Bregman and SME are absolutely unredeemable.

Accusations of racism and elitism

In interviews I've noticed that Bregman talks a lot about 'talented' people. For him, talented people are highly educated people. To get a fellowship with SMA, there are all kinds of criteria a candidate needs to hit and one of them is academic credentials, work experience and so on.

A picture does arise of a very elitist, and also tightly controlled organisation where the founders determine - based on criteria they pulled out of thin air - if you are a good candidate or not. And they alone decide what is moral, which topics should be spend time and effort on.

This is what Anita Naidu articulates in an article from 2025 where she states:

Let’s be clear about who it’s built for. SMA isn’t for people organizing at the grassroots, risking safety to confront systems of oppression. It’s for the already credentialed: Ivy League grads, McKinsey consultants, social entrepreneurs. People groomed to lead—just not to surrender power. The School doesn’t ask them to reckon with injustice. It teaches them how to perform reckoning without threatening their position.

She continues later on:

Instead of interrogating capitalism, colonialism, or white supremacy, SMA asks: how can we make these systems feel more ethical? How can we reframe domination as duty? Structural violence becomes a backdrop for personal growth. Justice becomes self-improvement. Revolution becomes résumé fodder.

Meanwhile, Payal Arora wrote a critical piece on Bregmans article (an adapted excerpt from his book 'Humankind' ) written in 2020. She critisizes Bregman for simplifying the story, leaving out important context about colonialism and racism, showing that the story is actually a counter-example to his hypothesis that people are good by nature³.

According to Payal in this article, because of her criticism, she got into a 'twitter war' with Bregman about her criticism.

I also found a critical article and a follow up article from (yet again) Tim Schwab about how Bregman handled the uncovering and critique on hist past racist remarks about Rihanna.

Closing words

At this point I can only ask the reader not to get involved with the School for Moral Ambition as I think SME isn't moral at all. I believe there is nothing of substance to SME and I don't see anything tangible this organisation has achieved with the money spent.

Disclosure: I have no affiliations with tobacco producers or processors, I'm not a wealthy person trying to evade taxation and I'm also not in any way affiliated with food or meat production. In short, SMA isn't working on anything that would affect me personally in a negative way.

Objectively verified by independent trusted third-parties ↩
The luddite story was never about the technology itself, but the actual social context and impact, as is with AI. ↩
not to say that this proves if people are good by nature or not, just that the story doesn't support Bregmans narative. ↩

I resurrected my Dutch movie review site from 2003

2022-06-09T12:00:00+02:00

Introduction

Between 2003 and 2006, I ran a Dutch movie review site called moevie.nl.¹ I built the site and wrote the reviews. It never made any money. It cost me money to host, and it cost me a lot of time writing reviews, but I remember enjoying writing reviews about films I liked.

The gimmick of the site was that the reviews had two parts. The first part is spoiler-free, just giving a recommendation with some context to make up your own mind. The second part contained a reflection of the movie, which included spoilers.

Even back then, the site didn't win any design awards (from archive.org - click to enlarge)²

I started building the site a few months after finishing college (IT) in 2002 as I felt inept and had little confidence. Building something tangible felt like a good way to build up and demonstrate skills. And I had something to say about movies.

Although moevie.nl did not help me gain employment as far as I know, it was fun while it lasted. At some point, I didn't get much joy out of writing movie reviews and I let the site die.

I did keep backups of the database the code and the pictures though. And now after 18+ years I decided to resurrect the site, including all (old) reviews.

Why resurrect a dead website gone for 16+ years?

Rebuilding the site was just a way to spend time, a small hobby project. Something to be bussy with. The second reason is some kind of misplaced nostalgia. I sometimes regret shutting down the site, wondering what could have been if I persevered.

Losing and regaining the domain

Back in 2006, my hosting provider (non-profit with just a few servers) abruptly stopped operating due to hardware failure ³ and I was forced to move my domain to another company. At that time, private citizens in The Netherlands could not register an .nl domain, only businesses could, so that was a bit of a hassle.

Not long thereafter however, I decided to let the domain expire. It was quickly scooped up by 'domain resellers'. Years later I decided that I wanted moevie.nl back, but the sellers always asked insane amounts of money.

In 2019, I visited moevie.nl on a whim. To my surprise it didn't resolve anymore, the domain was available! I quickly scooped it up, but I didn't do much with it for a long time, until now.

Rebuilding the site

I really wanted to preserve the aesthetic of moevie.nl as it was back then. Especially in the context of modern web design, it does stand out. As a sore thumb - but still - I had a goal.

Having the code and database dump is one thing, but it doesn't tell you what it actually looked like in 2003-2006. I could have tried to get the old (PHP4) code working, but I just didn't feel like it.

Instead, I chose to visit Archive.org and indeed, it captured old snapshots of my site back in 2006. So those were of great help. The screenshots at the top of this blog post are lifted from this page on archive.org. This snapshot was taken just before I decided to close the site.

The challenge of mobile device screens

To set the stage a bit: the rise and fall of moevie.nl happened a year before the iPhone was first announced. Smartphones from Blackberry were popular. I had a Palm VX PDA and later a HP Compaq PDA.

Most people didn't have mobile data connections so as far as I know, the mobile web really wasn't a thing yet.

So moevie.nl was primarily developed for the desktop. When I thought I was finished rebuilding the site, I quickly discovered that the site was unusable on my iPhone and way too small and finicky to use on my iPad.

For somebody who has no experience with modern web development, it was quite a steep learning-curve discovering how to deal with the various screen sizes in CSS⁴.

A very large part of the entire effort of rebuilding the site was spend on making the site workable on all different device sizes. Fortunately, iOS device simulators were of great help on that front.

Technology

I've recreated moevie.nl with Python and Django. For the database, I chose Postgresql, although that is total overkill, I could have used SQLite without any issues.

I chose Django because I'm quite familiar with Python so that was a straight-forward choice. I selected Postgresql mostly just to regain some knowledge about it.

Hosting

I'm self-hosting moevie.nl on the same Raspbery Pi4 that is hosting this blog. This Raspberry Pi is powered by the sun.

So moevie.nl is solar-powered during the day and battery-powered during the night.

Closing words

I'm not sure if I really want to start writing movie reviews again, knowing full well how much effort it takes. Also I'm not sure I have anything to say about movies anymore, but we'll see.

The overall experience of rebuilding the site was frustrating at times due to the severe lack of experience and knowledge. Now that the site is done and working, even on mobile devices, that feels good.

The name is based on the phonetic pronunciation in Dutch of the English word 'movie'. ↩
sorry for the language but I could not find a better screenshot. ↩
I was neglecting the site at that time due to losing motivation. ↩
I admit I only tested with iOS devices so Android-based smartphones could experience issues. ↩

Cryptocurrencies are detrimental to society

2020-03-27T12:00:00+01:00

Want to listen to this article?

Your browser does not support the audio element.

Introduction

How would you explain the inner workings of bitcoin to a person in simple, understandable terms?

source

This explanation seems perfect to me because it illustrates some seriously problematic aspects of cryptocurrencies in one simple sentence.

It captures the unimaginable energy waste of mining cryptocurrencies. And it also captures the dark side of cryptocurrencies: facilitating crime.

At this point cryptocurrency enthusiasts are rolling their eyes and sigh. This point has been made many times over. I know, I'm definitely not the first to criticise cryptocurrencies¹ this way.

I do have a simple challenge though:

Can you please show me the benefit to society of cryptocurrencies?

Please, don't come up with theoretical or future possibilities. Cryptocurrencies have existed for eleven years, they should have something to show for right now. After many hours of reading up on the topic, I have not been able to find any tangible benefits that would justify the effort and resources spend on them.

The downsides of cryptocurrencies are an entirely different matter. They are very, very clear to me. But let's not go there directly. What fun would that be?

The solution in search of a problem

As I see it, cryptocurrencies are entangled in a desparate search for a problem to solve. They are the answer to a question nobody asked².

Many cryptocurrency advocates see the decentralised (distributed) nature of cryptocurrencies as a tangible benefit. Cryptocurrencies are most often not controlled by any government or single entity.

Aside from whether this is true in practice³, they seem to imply that it's a bad thing that governments control their own currencies. Well, last time I checked governments control their currencies to keep them as stable as possible. Frankly, that actually sounds like exactly what I would want from a currency.

Stability.

You can call cryptocurrencies many things but 'stable' is definitely not one of them. Cryptocurrencies are highly volatile. In cryptocoin, a loaf of bread could suddenly cost twice as much as the day before.

Although cryptocurrencies haven't seen any significant adoption as a payment method - due to their volatility - it has seen adoption in 'less stable countries' where it is basically the 'lesser of two evils'. I mean: you know things are bad if a volatile cryptocurrency is a safer option than the native currency of your country.

The argument in the end boils down to: if your society is already in serious trouble, maybe cryptocurrencies could provide 'some benefit'. If the people involved have reliable access to internet. And internet access is most often controlled by the government.

To my knowledge, cryptocurrency as a payment method has actually only seen true adoption within the world of dark markets such as Silk Road in wich bitcoin rose to prominence. Nonetheless, after eleven years, cryptocurrencies have no traction in the regular 'legitimate' markets as a real payment method.

The reason why is obvious: existing payment methods are much easier to use and feel much safer. And the volatility of cryptocurrencies only compounds to the support of these conventional methods.

If cryptocurrencies are a solution to anything at all they seem to be 'bad' solutions at best.

Are cryptocurrencies in fact a Ponzi or Piramid scheme?

It depends on the particular currency, but I think the case can be made for sure. I mean: why not both?

Cryptocurrencies have no intrinsic value. They are only worth what people are willing to pay for them. So the cryptocurrency advocates needed to drum up demand, to create a market where previously none existed. This resulted in wild visions of the future, elaborate jargon-filled smokescreens that argue cryptocurrencies would take over the world. Get in quickly or you miss out!

And so many people were afraid to miss out, creating an enormous cryptocurrency hype, starting in November of 2017, spilling into 2018 when the bubble burst. The end result? A few people got very rich and the vast majority lost money.

It seems to me this all is a combination of a ponzi scheme with the component of active recruitment found in piramid schemes. The value of the currencies must come from somewhere, right?

So please tell me how all of this benefits our society? Creating a handful of rich people at the expense of a lot of other people? Is that it?

The graveyard of dead cryptocurrencies only shows how many people or startups try to get a piece of the action. And so many of them are outright scams. Are in fact all cryptocurrencies scams at their core?

The downsides of cryptocurrencies

I hope I have established that cryptocurrencies provide no tangible benefits to society. But the do have a lot of downsides. I observe the following:

Topic	Remark
Trafficing of illegal goods	Drugs, weapons, childpornography, and so on.
Trafficing of illegal services	Murder for hire
Tax evasion	-
Money laundering	-
Ransomware	Hold data hostage in encrypted form
Polution / energy waste	crypto miners use a lot of electricity
Lack of Security	several cryptocurrency exchanges have been hacked
Crypto scams	New cryptocoins are created just to scam people

The easy retort to this table is: "'normal' currencies like the dollar or euro also facilitate almost all of those illegal things", which is true but it misses the larger point.

Those regular currencies provide tremendous value to our societies. Our societies are build upon them and they facilitate almost everything we do. The topics listed in the table are just a possible negative side-effect for regular currencies. Their clear benefits outweighs such downsides.

Cryptocurrencies don't seem to have any such upsides. They seem to be made to exclusively facilitate cryptocurrency speculation and crime.

Cryptocurrencies facilitate crime

I won't discuss all the topics in the previous table but I do want to highlight a few.

Dark markets

I think we all remember Silk Road, a now defunct darknet marketplace that allowed people to anonymously buy - amongst other things - drugs and guns⁴. Silk Road was the first large-scale application of bitcoin as a means of payment.

Silk Road started out with just drugs, but guns soon followed. This deeply depraved world of dark markets are very much enabled by cryptocurrencies because the parties involved in a transaction are so hard to identify.

Ransomware

Ransomware seems to be almost exclusively enabled by digital currencies. Not explicitly just cryptocurrencies, but they do enable this type of crime because tracing the payments back to the criminal is so difficult.

The damage caused by ransomware is so obviously devastating. You can have many opinions on the fact that many critical organisations such as hospitals or universities don't have their computer security under control.

The real problem is that cryptocurrencies make these kinds of attacks on businesses and institutions very low-risk and highly profitable. In my own country a university was targeted by such an attack and allegedly they paid the ransom. The disruption to its services was substantial.

Energy waste

As cryptocurrencies rose in value, it started to become profitable to 'mine' them. It started out with regular computers, but soon, we could use videocards to accelerate cryptocurrency mining, which are very power hungry.

Later on, FPGAs and ASICS were build to further accelerate mining performance. Entire companies spun up to build those miners and host them in large datacenters with cheap electricity. The scale of the operation is rather enormous.

According to an article dating to July 2019, just bitcoin mining consumes more electricity than Switzerland. The article links to an online tool that tracks this power usage in real-time based on some estimates.

Image source

It's just mind boggling to me that so much energy is wasted, so much pressure is put on the environment, for absolutely no clear benefit at all.

Closing words

So in short, I think that cryptocurrencies provide nothing of value to society. They do however facilitate crime and contribute to climate change.

Therefore, I would propose to shut them all down.

The complex technology behind the cryptocurrencies attracted a lot of otherwise smart people and I think it's a sad thing to see their efforts going to waste or have a negative impact.

People are not obliged to work on something valuable, but at least may I ask that they choose to work on something that won't harm our society?

Link to hackernews, where this post was quickly flagged down. The few comments that exist don't seem to really provide any answers to the question I pose.

I would definitely recommend reading this long-form-article by The New York Times. ↩
Unless you want to embark on a path of criminal activity. ↩
As mining is no longer profitable for the larger community, the miners become a small concentrated group of entities controlling the currency, making the currencies more centralised. Furthermore, the cryptocurrency exchanges where you can convert the cryptocurrency into regular money, are centralised institutions backed by for-profit companies. And those companies have to abide by the law. They are under the influence of the government. ↩
If you want to know more about what happend to Silk Road and it's founder - 'the Dread Pirate Roberts', I would recommend the book 'American Kingpin' by Nick Bilton. (no affiliate links) ↩

FreeBSD 10.1 unattended install over PXE & HTTP (no NFS)

2015-01-16T12:00:00+01:00

To gain some more experience with FreeBSD, I decided to make a PXE-based unattended installation of FreeBSD 10.1.

My goal is to set something up similar to Debian/Ubuntu + preseeding or Redhat/CentOS + kickstart.

Getting a PXE-based unattended installation of FreeBSD 10.1 was not easy and I was unable to automate a ZFS-based install using bsdinstall.

I would expect someting like the netboot install

Below, I've documented what I've done to do a basic installation of FreeBSD using only DHCP, TFTP, no NFS required.

Overview of all the steps:

have a working DHCP with PXE boot options
have a working TFTP server
customise your pxelinux boot menu
install a FreeBSD box manually, or use an existing one
download and install mfsbsd on the FreeBSD system
download a FreeBSD release iso image on the FreeBSD system
configure and customise your FreeBSD PXE boot image settings
build the PXE boot image and copy it to your TFTP server
PXE boot your system and boot the FreeBSD image

Setting up a DHCP server + TFTP server

Please take a look at another article I wrote on setting up PXE booting.

Configuring the PXE boot menu

Add these lines to your PXE Menu:

LABEL FreeBSD10
kernel memdisk
append initrd=BSD/FreeBSD/10.1/mfsbsd-10.1-RC3-amd64.img harddisk raw

Setup or gain access to a FreeBSD host

You need to setup or gain access to a FreeBSD system, because the mfsbsd tool only works on FreeBSD. You will use this system to generate a FreeBSD PXE boot image.

Installing mfsbsd

First we download mfsbsd.

fetch http://mfsbsd.vx.sk/release/mfsbsd-2.1.tar.gz
tar xzf mfsbsd-2.1.tar.gz

Then we get a FreeBSD ISO:

fetch http://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/FreeBSD-10.1-RELEASE-amd64-disc1.iso

Mount the ISO:

mdconfig -a -t vnode -f /root/FreeBSD-10.1-RELEASE-amd64-disc1.iso
mount_cd9660 /dev/md0 /cdrom/

setup rc.local

Enter the mfsbsd-2.1 directory. Put the following content in the conf/rc.local file.

fetch http://<yourwebserver>/pxe/installerconfig -o /etc/installerconfig
tail -n 7 /etc/rc.local > /tmp/start.sh
chmod +x /tmp/start.sh
/tmp/start.sh 
exit 0

#!/bin/csh
setenv DISTRIBUTIONS "kernel.txz base.txz"
setenv BSDINSTALL_DISTDIR /tmp
setenv BSDINSTALL_DISTSITE
ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/10.1-RELEASE

bsdinstall distfetch 
bsdinstall script /etc/installerconfig

As you can see there is a script within a script that is executed separately by rc.local. That's a bit ugly but it does work.

setup installerconfig (FreeBSD unattended install)

The 'installerconfig' script is a script in a special format used by the bsdinstall tool to automate the installation. The top is used to control variables used during the unattended installation. The bottom is a script executed post-install chrooted on the new system.

Put this in 'installerconfig'

PARTITIONS=da0
DISTRIBUTIONS="kernel.txz base.txz"
BSDINSTALL_DISTDIR=/tmp
BSDINSTALL_DISTSITE=ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/10.1-RELEASE

#!/bin/sh
echo "Installation complete, running in host system"
echo "hostname=\"FreeBSD\"" >> /etc/rc.conf
echo "autoboot_delay=\"5\"" >> /boot/loader.conf
echo "sshd_enable=YES" >> /etc/rc.conf
echo "Setup done" >> /tmp/log.txt
echo "Setup done."
poweroff

As you can see, the post-install script enables SSH, sets the hostname and reduced the autoboot delay.

Please note that I faced an issue where the bsdinstall program would not interpret the options set in the installerconfig script. This is why I exported them with 'setenv' in the rc.local script.

With Debian preseeding or Redhat kickstarting, you can host the preseed or kickstart file on a webserver. Changing the PXE-based installation is just a matter of edditing the preseed or kickstart file on the webserver.

Because it's not fun having to generate a new image every time your want to update your unattended installation, it's recommended to host the installerconfig file on a webserver, as if it is a preseed or kickstart file.

This saves you from having to regenerate the PXE-boot image file every time.

You can still put the installer config in the image itself. If you want a fixed 'installerconfig' file containing the bsdinstall instructions, put this file also in the 'conf' directory. Next, edit the Makefile. Search for this string:

.for FILE in rc.conf ttys

For me, it was at line 315. Change it to:

.for FILE in rc.conf ttys installerconfig

Building the PXE boot image

Now everything is configured, we can generate the boot image with mfsbsd. Run 'make'. Then when it fails with this error:

Creating image file ...
/root/mfsbsd-2.1/tmp/mnt: write failed, filesystem is full
*** Error code 1

Stop.
make: stopped in /root/mfsbsd-2.1

just run 'make' again. In my experience, make worked the second time, consistently. I'm not sure why this happens.

The end result of this whole process is a file like 'mfsbsd-se-10.1-RC3-amd64.img'.

You can copy this image to the appropriate folder on your TFTP server. In my example it would be:

/srv/tftp/BSD/FreeBSD/10.1/mfsbsd-10.1-RC3-amd64.img

Test the PXE installation

Boot a test machine from PXE and boot your custom generated image.

Final words

I'm a bit unhappy about how difficult it was to create an PXE-based unattended FreeBSD installation. The bsdinstall installation software seems buggy to me. However, it could be just me: that I have misunderstood how it al works. However, I can't seem to find any documentation on how to properly use the bsdinstall system for an unattended installation.

If anyone has suggestions or ideas to implement an unattended bsdinstall script 'properly', with ZFS support, I'm all ears.

This is the recipe I tried to use to get a root-on-zfs install:

ZFSBOOT_POOL_NAME=TEST_ROOT
ZFSBOOT_VDEV_TYPE=mirror
ZFSBOOT_POOL_SIZE=10g
ZFSBOOT_DISKS="da0 da1"
ZFSBOOT_SWAP_SIZE=2g
ZFSBOOT_CONFIRM_LAYOUT=1

The installer would never recognise the second disk and the script would get stuck.

I'm aware that mfsbsd has an option to use a custom root-on-zfs script, but I wanted to use the 'official' FreeBSD tools.

Finding a good blu-ray player for Mac OS X

2013-09-22T18:00:00+02:00

I find playing a Blu-ray movie on my Mac cumbersome. I've been using Plex, XBMC and VLC but these free open-source products are all a usability nightmare.

To play a Blu-ray movie, you have to perform these steps:

right-click on the BDMV file
choose 'show packet contents'
go to the STREAM folder
sort the files by size (from large to small)
select the biggest m2ts file and open it in the appropriate player

I got so fed-up with this process that I started searching for any product that just allows me to point it to a folder with Blu-ray content and friggin' play it. Fortunately, there is such a product for Mac OS X, it's called Mac Blu-ray Player and it's a paid application. I paid 36 euro (48 dollar) including taxes.

Those 36 euros are well spend. No it is not free, no it is not open-source, but I don't care. It is a good product. If you value your time I highly recommend buying this software.

I don't have anything against free or open-source software, but I do have a grudge against software that is not user-friendly. If software is not easy to use, something you would expect from a media player, it's broken.

Fortunately, you don't have to trust me on my word, you can download a free trial that seems fully functional, it only shows a trial message when playing a movie.

I switched my blog from Blogofile to Pelican

2013-08-06T03:00:00+02:00

This blog is a static website, which makes it fast, simple and secure. It was generated by Blogofile but I switched to Pelican.

Blogofile has seen almost no updates over the years and I consider the project dead. Realising that blogofile is dead, I decided to look around for different open source static blog generators.

There are many other static blog generators than pelican. But Pelican is well-documented, is based on Python, is very actively maintained (good track record) and supports all features that I wanted. Some of those features are Atom/RSS, Disqus and Google analytics support.

My blog posts are written using Markdown. This makes it very easy to migrate away from Blogofile to Pelican, as Pelican also supports Markdown. Blogofile uses a different header format not recognised by Pelican, so you have to search and replace some key words in all your files before Pelican can actually generate your new website.

I wrote this horrible bash shell for-loop to process all my blog posts:

    :::bash
    for x in *.md
    do
        TITLE=`grep -i "title:" "$x"`
        TITLEFIXED=`echo $TITLE | sed s/\"//g`
        DATE=`grep -i "date:" "$x"`
        DATEFIXED=`echo $DATE | sed "s/\//-/g" | cut -d ":" -f 1,2,3`
        CATEGORY=`grep -i "categories:" "$x"`
        CATEGORYFIXED=`echo $CATEGORY | sed s/\"//g | sed s/categories/category/g | cut -d "," -f 1 | /usr/local/bin/sed -e "s/\b\(.\)/\u\1/g"`
        echo "$TITLEFIXED" > tmp.txt
        echo "$CATEGORYFIXED" >> tmp.txt
        echo "$DATEFIXED" >> tmp.txt
        grep -v "title:" "$x" | grep -v -e '---' | grep -v -i "date:" | grep -v -i "categories:" >> tmp.txt
        mv tmp.txt "$x"
    done

Notice how Build-in syntax highlighting of Pelican applies nice colors to this horrible code. Regarding this horrible code: I had to use GNU sed as the Mac OS X sed did not support the regular expression I used.

To enable comments for my blog posts I always used Disqus with Blogofile. Pelican generates web pages in a different way compared to blogofile, so all old pages need to be redirected to the new location. I used the redirect functionality of Lighttpd to redirect all existing pages to the new location.

The cool thing is that Disqus has a tool called "Redirect Crawler". If you have configured 301 "permanent redirects" for all pages and run this tool, Disqus will automatically update all existing links to the new locations, so your comments are migrated to the new web pages.

Furthermore, I've implemented a Pelican plugin called titlecase which capitalizes the first letter of words in the title of your article. It's just that I think it looks better.

I think I'm really happy with Pelican.

Why I believe the new Mac Pro won't be a great machine for gaming

2013-06-23T00:00:00+02:00

In Accidental Tech Podcast episode 18 (love the show), I learned that John Siracusa was thinking about buying a new Mac Pro for gaming.

I believe that gaming on the new Mac Pro will be a mediocre experience.

Driver support: as John mentioned himself, the video cards are 'professional GPUs' used in workstations for computing, CAD etc. These cards and especially their drivers under Windows are not geared towards gaming performance.

The performance and quality of Mac OS X drivers may be improved dramatically over the past years, but if you like to play games exclusively under Windows, you will probably be disappointed when you switch to bootcamp.

Crossfire support: you have these two ridiculously fast GPU's and for years on the PC platform, you can stack them together to achieve insane performance (SLI / Crossfire).

Mac OS X does not seem to support crossfire (or SLI). If you can't benefit from crossfire, you're paying a lot of money for a machine with one idle but very expensive videocard. That sounds like a ridiculous waste of good money.

Assuming that the hardware supports crossfire, if you would run Windows, then you may hit the driver issues associated with pro cards.

Upgrading: upgrading can be cost-efficient as most games are GPU not CPU bound, not an option for the Mac Pro. Even the new Thunderbolt interface does not have sufficient bandwidth to hook up one or more external high-performance videocards, aside from the fact that this will be insanely expensive.

If you really needs the horsepower of a Mac Pro for other purposes than for gaming, sure it's the fastest mac you can get, but otherwise, I believe there's a better deal to be had.

We don't now what the new Mac Pro will cost in a configuration suitable for gaming, but it will be 'a lot'. I believe that for one new Mac Pro, you will also be able to buy:

a mac Mini with reasonable specs (it's easy to replace memory and storage);
a high-quality 27" display @ 2560x1440
a ridiculously fast PC that will outperform the new Mac Pro when it comes to gaming.

I know that some Mac users don't like the idea of having a PC at home, but if you are into PC gaming, there's no other choice in my opinion if you want a good gaming experience.

I had high hopes for my 27" iMac (2011) but gaming performance anno 2013 is just mediocre at best. And I can't use it as an external display with a PC, only with a Mac.

So I sold my 27" iMac and used the money to buy a separate 27" display and Mac Mini. I also ordered a 'ridiculously fast PC' which I hope will allow me to play all modern games on max quality settings for now and the upcoming year. And if required, I can swap out the dual GPUs and replace them with something better over time, if I need to.

Why VMware vSphere replication is changing the game

2012-11-12T23:00:00+01:00

If you are running a serious VMware environment, chances are you do have a SAN. Often with smaller setups, many people do employ multiple VMware hosts, but the SAN is a single point of failure.

SANs are often fully redundant devices, with redundant PSUs, storage controllers, network links and RAID arrays. But with all that redundancy build in, they still fail. I've seen it happen and a failing SAN is the worst.

So I'd rather have two cheap entry level SANs if possible than just a single big one and keeping my fingers crossed that they won't fail.

A redundant SAN environment where you basically deploy two separate SAN devices with their own storage needs a replication setup. And replication between SAN environments needs often extra licences or more expensive setups. And the replication mechanism must play well with your virtualisation layer, such as VMware.

But the fun thing is that VMware made everything way simpler by integrating their own storage replication into their vSphere product. I have no experience whatsoever with VMware's new build-in replication feature. But I believe that it is significant.

Replication is a new feature introduced in VMware vSphere 5.1 that is now part of the vSphere Essentials Plus Kit and vSphere Standard. So if you start with two or three VMware hosts and two entry-level SAN devices, you can be quite redundant and can have a fully redundant setup. And that will cost you around 3800 Euro ~ 4800 US dollar.

The Essentials Plus Kit is a nice environment for smaller companies, but license-wise, it does not scale as you are stuck with a maximum of 6 physical CPUs and a maximum of three hosts. However it seems that when you need to expand beyond that capacity, you can trade in your existing license and obtain a discount when upgrading to - for example - vSphere Standard or Enterprise.

The most significant thing about the build-in replication is that it does not matter any more what you use for your storage backend. If you use two entirely different devices from different vendors, that's OK. Because VMware handles all the replication stuff. Those SAN devices become just dumb storage boxes. Most of us just can configure whatever does support iSCSI.

You could even try and be cheap and setup your own homegrown storage boxes. It may not have all the cool features of a true SAN, but at least you have redundant storage.

I'm really curious about this feature and I hope it works well. I'm seriously considering deploying this for the VMware setup of the company I currently work for. It does however require an extra external host that manages the actual replication, which may add to the cost.

Any comments are welcome.

Why I bought a digital projector (Panasonic PT-AT5000E)

2012-10-21T00:00:00+02:00

I don't have a TV. I haven't been watching TV for more than 10 years. But I love to watch movies or great series like Dexter and Game of Thrones. Until recently, I watched movies or series on my 27" iMac. Twenty-seven inch is large for a computer screen but for a TV, it's quite small.

When it comes to watching movies, a bigger screen is always better. The big screen of a movie theater is the most important reason for movie enthousiasts to experience a movie in the theatre. But there is a way to get that movie theatre feel into your living room: get a digital projector. That's what I did.

I decided to get a ceiling mounted projection screen that I could lower just in front of my computer and table. I chose the "celexon Rollo Economy with a dimension of 277 x 156 cm (109 x 61 inch). This would give me a TV screen with a theoretical size of 125 inch in diameter! In practice, the actual projected image is around 120 inch.

Just think about that. A 120 inch television. Hundred-and-twenty inch. Three metres diagonal.

The screen can be lowered by just pulling on the edge. I have to reach for it on my toes but it's ok. As pictured below, the screen has an additional black area so it can be lowered further away from the ceiling. This way, the effect of the ceiling reflecting light back to the screen, reducing picture quality, is reduced. It allows you to have the screen at the desired hight, you don't need to look up.

At the other end of the room, I decided to mount a projector on a ceiling mount. My iMac would sent all digital content through a long HDMI cable to the projector.

After doing some research I decided to buy the Panasonic PT-AT5000E also known as the PT-AE7000. This digital projector supports Full HD 1080p at 1920x1080. It also supports 3D but I'm not really interested in that.

I have never seen a movie on a digital projector before and I can't compare it to other digital projectors. But one thing is sure. Watching a 1080p movie on a 120" display projected by this digital projector is quite an experience. To me the picture is astounding. It is almost if you are in a theatre. A digital projector may not always be practical, but it is surely awesome.

I do have some remarks about the whole setup with a digital projector:

you need a big screen and mount it somewhere;
the projector is just gigantic and you must accept that it hangs from your ceiling somewhere, something your spouse may not like;
you need to run some extra cables for power and HDMI through your house;
you need to put the lens cap on when the device is not in use, often a hassle;
the lamp takes several minutes to reach optimal operational temperature;
although my panasonic projector is almost silent, it does make noise;
you must have good curtains to block all external light or you won't see a thing;
you need to wait for movies to be released on Blu-ray or the Internet to be able to watch them;
the lamp will wear out and replacements are expensive;
if anyone would ask you about the size of your TV you would have to lie or feel a douchebag for telling the truth.
Non-HD content looks totally crap and you need to make the picture smaller just to stand it.
the whole home theatre setup is just expensive and you could have bought a lot of theatre tickets for that money;
black levels are often not as good as plasma TV's.

But in the end I would not hesitate to fork out the money for a home theatre setup like this. Because of the downside of movie theatres:

other people. They are the cause that you don't have the best seat in front of the screen;
more other people, who text, talk or phone during the movie;
even more other people, who eat crappy food, making noise and drink crappy liquids as if it where a slurping contest;
you need to pay money;
you need to sit in some crappy movie theatre chair;
you must pay top dollar for a drink or food;
no pauses so no toilet breaks;
you need to go there, and be there at a specific time;
you need to stand many commercials before the movie actually starts;
the sound quality is often horrible.

So in my opinion, if you you are willing to spend money on it, I would advice to seriously look into getting a digital projector.

Compiling Multicore PAR2 on Ubuntu 12.04 LTS Precise Pangolin

2012-09-16T01:00:00+02:00

If you want to compile PAR2 with multicore support on Linux, it may not work right away from source. I used this version of PAR2 with multicore support. Update 2015: the original link is dead, I foud a copy of the source and put it on my own site here for you to download.

First, make sure that you have these libraries on your system:

libtbb-dev
libaio-dev

According to this source, after downloading the source, you need to add this line:

#include <backward/auto_ptr.h>

To these files:

par2cmdline.cpp
commandline.cpp
par2creator.cpp
par2repairer.cpp

Then edit the Makefile and find the LDADD variable. Add the -lrt option like this:

LDADD = -lstdc++ -ltbb -lrt -L.

This did the trick for me and compiled the PAR2 source with multicore support. Hope it helps somebody.

If you want to use Multicore PAR2 with SABNZBD you need to go to the Config menu. Then select 'Special' and enable 'par2_multicore'. Save the changes. Then go to 'Switches' and enter '-t+' at the Extra PAR2 Parameters field.

Fully unattended deployment of Windows clients using limited resources

2012-07-07T19:00:00+02:00

Introduction

Anyone who ever installed Windows on a computer by hand must have wished for a solution that automate this task. It's just waiting a lot and pressing a button now and then. But installing the operating system itself is only the beginning. Once installed, you need to apply service packs or at least about a hundred or more security updates. When finished, you need to install all additional software, like an office suite, PDF reader, anti-virus software and the like.

So you need to install:

the operating system
applications
security updates

If you do this all by hand, it will probably take at least half a day, if not even longer. This is a major problem, because sooner or later you may have to hire somebody full time to do just the computer deployments. Expect a high job turnover rate. You definitely want to automate this task, saving money on extra sysadmins but more importantly: quality.

Even if you have to install one computer every week, you must automate this process for the sole reason that if you don't, no two deployed computers are the same. People make mistakes, especially with boring, repetitive tasks. So automation improves quality and reduces the workload significantly.

If you don't deploy your end-user computers through some kind of automation, you need to stop what you are doing right now and build such a solution. It's fundamental to provide good quality service to your users.

It must be fully unattended or as unattended as possible. You may have to press a button to initiate the process at that start, but that must be all that is required to deploy a system. If during deployment, you need to touch the computer in order for it to continue deploying, you have a bug that needs to be fixed asap.

So, in this post I want to show you that with minimal resources, you can create a fully unattended solution for Windows desktop systems. There are probably better ways to do this, but for me, this was enough.

Imaging versus automated deployment

It's very simple. Do not image. Do not use products like Norton Ghost or Clonezilla for system deployment. Imaging is not flexible. For every change, you need to create a new image. For every hardware model, you need to create a new image. Every program update requires a new image. Instead of installing computers by hand, you are maintaining images. It does not scale.

Automated installations on the other hand do scale. They are dynamic. They just use whatever drivers they need during installation, as long as they are available. Just updating the installer of an application is sufficient to make sure that future deployments are up-to-date. Flexibility is key.

Solution overview

Clients use PXE to boot from the network. They boot a special Windows Embedded kernel that bootstraps the Windows installation process.
The operating system and drivers are installed.
All company software is installed.
All security patches are installed.
When ready, a mail is sent to the sysadmins

You will need:

A DHCP server
A WDS server
A KMS server and valid KMS licence
Valid Windows 7 ISO for KMS installation
An unattended configuration created with WAIK
Driver packs for the various desktop an laptop models
A domain account dedicated for deployment
A list + executables of all software required for the client

About KMS and Windows licences

In a larger environment, with 25+ desktops and laptops, it becomes to cumbersome to type in the product licence key and activate the systems by hand. This does not scale. You need a Volume Licence agreement for Windows 7 or higher in order to be able to use a Key Management Server and a special ISO of Windows 7 that does not require a product key. Learn more about this in this blogpost.

Windows Deployment services

The basis for automated deployment is Windows Deployment Services. This software made available for free by Microsoft allows clients to PXE boot and perform unattended operating system installations.

Unattended operating system installations are guided by XML files that describe the configuration for the operatings system. Such a configuration file is authored with the Windows Automated Installation Kit.

WDS uses two images: a boot image and an install image. Because computers need drivers, you need to download and inject the drivers in the boot image. All major vendors supply special complete driver packages for you to download. Just download, extract and import. Create driver groups for every model, to orden your drivers.

You may choose to install all drivers in one image. But that image can grow large and lengten the installation time. To resolve this, create separate boot images for different vendors and differentiate between model lines. This is not much work but it keeps the boot images small. This is not required.

Windows Automated Installation Kit

You need the WAIK to author the XML file used by WDS to configure the unattended installation. You must specify hard disk partitioning, some default settings and the like. This is also where you configure the command to run when the operating system installation has finished. This will start the software installation phase.

Automated silent software deployment

For software installation, I just go back to my MS-DOS 4.11 days and use a simple batch script that installs all software.

Every product, such as Adobe Reader or Java, has an installation batch file. There is one main batch file that calls each program install batch file to install it and log the results for debugging.

It is that simple. And it works perfectly. The most important task is to find out for each product how you can install it silently, without user intervention. Fortunately, almost all products provide command line arguments for unattended installation.

Software is installed by using a domain-based unprivileged user that uses autologon to logon to the system, with local administrative privileges. Once the installation is complete, local admin privileges are revoked.

Installing all security updates

This is the hard part. There are several problems. First, after you install all updates, more updates seem to be available after the next reboot. Furthermore, using Windows 7, a memory leak causes the installation proces to take ages.

The solution is to install smaller batches of patches, such as 30 or 40 at a time. You can use a script for that as supplied by Microsoft. This script must be changed not to install all patches, but a fixed number at a time.

So you need several reboots to install all patches and run the VBS update script several times. The WAIK provides an option for 'autologon'. So you can have a user account logon for like 5 times. After that, no autologon is performed ever again.

So you place a special batch file in the startup folder of the autologon user that triggers the Windows update process every time the autologon is performed. This is the last step of the installation.

After five autologons, the system will boot to the logon screen and the system is done.

Additional resources

Large organisations may use Microsoft System Center Operations Manager but I assume that such a solution has not been setup. I asume, that you are in an environment without any existing solution that may help you out.

I would also investigate the Microsoft Deployment Toolkit 2012. Instead of tinkering with batch files and vbs scripts, this may help you also. However, it seems to focus on creating images or automate the task of creating images, rather than just automate the installation of a client.

Final thoughts

Please note that I had to research this solution within a few weeks, with lots of other things to do. It was just one project of many other projects. There may be better solutions to automate system deployments. Maybe the MDT is a better approach, but I haven't tested it (yet). The current setup is sufficient for now and it frees us to start other much needed projects.

Understanding Windows KMS and MAK volume license activation

2012-06-09T21:00:00+02:00

Introduction

If you have to administer a large number of PCs running Windows, you will end up creating an automated deployment platform for your Windows clients. You may implement something like Windows Deployment Services.

I used WDS to create a fully automated installation of PCs. WDS can also be used for creating images, but using images doesn't scale as you need too much manual intervention with the devices themselves and you need to update images constantly.

With WDS and some driver packs I can support as many different computer brands and models as I want with a single vanilla Windows 7 base image. All customization and automation is done with answer files using the Windows Automated Installation Kit.

When creating an automated deployment environment, one thing you definitely don't want to be doing is having to enter each individual Windows product key as found on the sticker somewhere on the chasis. You want a single key, embedded in the deployment image or script and run with that, or some other solution. Your goal must be to do away with manual product key input and activation.

This is not a problem, but here we have to introduce the topic of licences, especially client licences such as Windows 7. There are only two flavors of Windows:

Retail - this licence is most expensive but allows you to transfer it from one computer to the other one.
OEM - this licence cost you less but is tied to that particular computer.

The important thing for a system administrator to know is this: when buying OEM, you do not have rights to create disk images or do something similar with these computers. You cannot use Windows Deployment Services, You cannot use cloning tools or other solutions.

Volume licensing

Now it is time to talk about volume licensing. A volume licence is an upgrade of a Retail or OEM license. So one thing is sure: you must order every computer with an OEM Windows licence, regardless of your plans. In addition, the volume licence, you have to buy separately, you gain 'reimaging rights'.

Now comes the fun part. You only need one (1) Volume Licence for a specific product to be eligible to image or automatically deploy all PCs running that particular operating system (32 bit or 64 bit doesn't matter).

KMS or MAK activation

With a volume licence, client's don't need to activate with Microsoft through the internet. For larger organisations, that would cause too much internet traffic. Instead, you use a local activation service within your network. You can either deploy a KMS (Key Management Service) or use the Volume Actication Management Tool (VAMT).

Most people may want the KMS service. But a KMS service only starts to validate clients with 25 or more PCs. If you have less than that number of clients, you may resort to MAK validation.

When choosing KMS activation, you install a KMS service on one of your Windows servers and that host will then act as an activation server within your organisation. Systems activated through the KMS must periodically revalidate themselves (as like every 6 months / 180 days). but how do the clients know that they should validate against your KMS? And which product key do you use?

If you buy a volume licence, you will get access to a special ISO image of Windows 7, Vista Business or XP Professional. You also gain access to a special product key, a KMS product key. (Please note that you must by a volume licence for each operating system product version).

You use this special KMS product key to activate the KMS server. This happens only once. So this one time, you activate the KMS server with Microsoft, after that, no communication occurs with clients or the KMS service with Microsoft.

That special ISO image you got contains a special Windows version that does not require a product key. Once a client is installed, it just searches your network for a KMS server through DNS and tries to activate against it. Once validated, clients stay validated as long as they get in contact twice a year (180 days) with your KMS service.

If you have less than 25 PCs, you will use the MAK activation and the VAMT tool. Clients can either activate through Microsoft directly or through the VAMT tool. The VAMT tool collects activation requests within your network like a KMS, however, it does contact Microsoft to validate those activations. And there is a limited number of activations you are entitled to. This VAMT tool can cache activation requests so you can redeploy or re-image systems and reactivate them without seeing your activation limit getting reached.

I hope this information was useful to you and if you've discovered a mistake, please comment.

Neato XV-15 / XV-11 Robotic Vacuum cleaner review

2011-12-25T09:00:00+01:00

Update 18 February 2012

There is one problem. When the robot is not connected to the charger, the batteries are depleted very fast. Even if the batteries are not entirely depleted and the robot can still display the menu, the clock loses it's time. Every time the robot gets a too low charge, you have to set the date and the time, which is a bit of a hassle. This does not happen often though. The robot seems to be consistently operating properly.

Update 1 March 2012

It seems that the batteries have worsened so bad that the device cannot clean my living room without 3x recharging. I have to return the product for repair. I had the device scheduled to clean every other day, about 4x per week.

Update 20 March 2012

I received a brand new device that is now charging. I hope this one will last longer.

Update 23 March 2012

It seems that the brand new robot is also flawed, it just goes nuts. Seems to be up-to-date regarding software, so have to return this one also. (read below!)

Update 31 March 2012

I did not return this device and did some additional cleaning cycles. All cylces where performed withouth problems. The device choked om some cloth and some cables I forgot to cleanup, but it does seem to operate properly. So I will keep it.

Uodate 4 May 2012

Still works like a charm. I'm currently very hapy with it. If the batteries hold up, this device is really worth the money.

Uodate 8 June 2012

I had some critical battery errors and contacted support. They asked me to check if the batteries are connected properly. So I just pushed on the connectors to make sure they are firmly connected. After that, I didn't see any more battery errors and the device is still cleaning like a charm.

Original article:

So I bought a robotic vacuum cleaner. The first question is 'why would you spend some serious money on such a device? On a toy?'. I have some rationalisations for buying this device, but honestly, one reason is that sometimes I just like to buy a new toy. Something to play with. Excuse me for being human. In this blog post I want to explain to you why I bought a Neato XV-15 and not another product.

Now I did say that I have some rationalisations, so let's start. One rationalisation is that I hate vacuum cleaning. Since I have two cats, vacuum cleaning once a week may not be enough. And I'm not going to clean more frequently. So you can accept it or if you can spare a little dough, buy a robotic vacuum cleaner that cleans your house when you're not at home.

So let's introduce the Neato XV-15.

The Neato XV-15 Vacuum cleaning robot

The XV-15 robot is made by Neato Robotics, a young startup that seems to be started purely for this device. The company started with the XV-11 for the US market, and the XV-15 is identical except that it is meant for the European market. A new XV-12 has also been announced, which seems to be identical to the other two machines, except for the color (white).

The robot automatically vacuums your house while you're away or minding your own business. I't can't do anything else, but not having to vacuum all the time is kinda cool, right?

I bought the XV-15 in The Netherlands for 500 euros. The XV-11 can be had for around $400 excluding taxes or maybe even for less at Amazon. Not very cheap, but competitively priced compared to other robots on the market.

How the robot works

The XV-15 has a rubber brush at the front that rotates quite fast and that brush scoops up the dirt. Just behind the rubber brush, a vacuum mouth is present. Anything sucked up through that mouth enters the dustbin. The actual vacuum motor is at the back of the dustbin, protected by the dust filter of the dustbin. The XV-15 is a true vacuum and Neato claims that vacuuming power is way stronger than any other robot on the market. Based on the noise, that may be true.

On top of the XV-15 you can find an LCD screen for configuring the robot and the turret housing its special secret weapon: laser sight. This is the cool part.

The XV-15 has a laser system mounted on top that allows the robot to locate objects and walls. It is capable of creating a map of its surroundings. Anything the laser can 'see' will be avoided. The robot will not bump into any objects it can see. This is in stark contrast to products like the iRobot Roomba, which just bumps into everything. The XV-15 does have a front bumper though, because anything below the laser turret cannot be seen. Thus the robot does bump into things occasionally but it does a hard job trying not to.

The laser system is not just for preventing collisions with furniture. Being able to generate a map of your house allows the robot to clean your house in an efficient manner. Robots like the Roomba just randomly zigzag through your house. If you do that long enough, chances are high that most of your house gets cleaned, which it will.

The XV-15 only covers each spot once, and thus is able to clean your house much faster. It first cleans the perimeter of a room, hugging the walls. It then cleans the room in straight lines, like a swimmer in a pool. It remembers where it has cleaned or not and will come back later to a spot if something (like humans or pets) was occupying an area that can now be cleaned.

My living-room, kitchen and entrance are cleaned in 40 minutes. An area of 40 square meters or about 420 square feet.

When you see the XV-15 doing it's job, you may tend to stare at it longer than you may want to. It's just fascinating to see the device effortlessly navigating around your house. And it doesn't need stuff like battery operated 'light houses' like the Roomba's need. It is truly autonomous except for emptying the dust bin.

The XV-15 seems to divide the rooms it detects in parts and will start cleaning those parts one after another. As said earlier, the robot will continue cleaning where it had left off if the batteries are low and needs recharging.

The robot has no problem detecting stairs. Neato has also provided a roll of magnetic strip that can be used as a boundary marker. The robot will not cross this strip and will clean around it.

However, how smart the XV-15 may be, you need to make your house robot-proof. The first time you start cleaning with the Neato, it is advised to monitor it's progress and 'fix' difficult spots in your house. I have no experience with other robots, but I think that this is true for all of them.

The robot is just low enough that it can clean underneath my central heating radiators, which is very nice. It also has no trouble cleaning under my bed, an area which seems to collect dust very fast.

The robot has never had any problems finding the base. It gently wiggles it's behind towards the base until it has a connection. It then informs you with a sound that it has finished cleaning.

Docking station

The XV-15 comes with a docking station that allows the device to automatically recharge for the next run. The XV-15 will return to the docking station if the batteries are low. When recharged, the XV-15 will continue cleaning where it left off. If you have a single story apartment, the XV-15 will thus clean the entire apartment all by itself, even if it can't clean your home in one take on a single battery charge. After recharging, the unit will just return to the spot where it aborted cleaning to recharge and continue cleaning.

The docking station allows you to put excess power cord into the station itself, to keep cable clutter to a minimum. You can also reroute the cable to exit the station from either the left side or right side.

Scheduling

The robot can start cleaning with a press of the big orange button. The robot will start cleaning and return to the docking station when finished. Ideally, you want to have the robot clean the house when you're not around. Fortunately you can set a schedule for all seven days of the week.

The robot has a clear LCD screen with a very easy menu for setting the clock and entering a schedule. A few simple buttons allows you to enter a schedule, which probably has to be done once. I have it set to clean every other day except for the weekend.

Scheduling is extremely simple: for all seven days of the week, you can configure a start time or choose not to clean that day. That's all.

Noise level

When you start the XV-15 for the first time, you will be surprised by the of noise this little device generates. The vacuum motor is loud, but the rubber brush adds an additional roaring and rattling sound to it that is just almost unbearable.

The rubber brush keeps hitting the floor causing the loud rattling sound. I had to add some felt strips on the bottom to raise the robot a little bit from the ground. This eliminated the rattling, but the robot is very loud. Keep this in mind.

I think the noise level is the biggest downside of this robot.

Cleaning performance

The picture shows what the XV-15 can collect during a sweep. I dit not perform any scientific tests to verify the cleaning performance of the robot, but any visible dirt is always devoured by the robot. I'm personally very pleased with the results.

I found a source written in Italian that seems to suggest that the XV-15 does a significantly worse job of cleaning stuff (67%) than the Roomba 780 (97%) robot, but it is an artificial test that does not use the stuff it is supposed to clean: (fine) dust and hair. However, it thus may be possible that the dumb Roomba's clean better. I don't know.

I only can tell you that even if you clean daily and you have some pets, you will find quite some stuff inside the dustbin after each run.

Maintenance

The iRobot Roomba range of products seem to require quite some maintenance. The biggest issue with the Roombas is the fact that you need to clean out hair from the bearings and brushes after each run. This is not necessary with the XV-15.

I don't know how much time cleaning of a Roomba takes, but I have an issue with that: why bother with a robot if you have to clean the robot instead of the house itself? Yes cleaning the robot takes less time, but it's probably no fun either.

The only thing that you need to do when the XV-15 is finished: empty the dustbin and clean the filter. That will take no longer than 30 seconds I guess. No need to clean up the brush or bearings. It is of course advised to inspect the brush and bearings now and then.

Checking the condition of the rubber brush and bearings is very easy. The brush guard can be removed without tools in seconds. Removing the rubber brush is just as easy and cleaning the axles shouldn't take long if ever required. I've never had to clean the brush itself. It seems that hair gets sucked up and doesn't stick to the brush.

Inside the box

The XV-15 comes with an additional rubber brush and four additional filters. According to Neato, you need to replace the filter every three to six months, depending on the frequency of your cleaning schedule. At 16 euros ($20) for 4 filters, that's not a big deal I guess.

I couldn't find any details on how long the rubber brush will last.

Updating the software

If you take a closer look at the back of the robot, you will notice that at the left side of the big exhaust vent, two small ports are present: for power (only useful if you do not use the docking station) and a USB port. The USB port can be used to update the software of the robot to the latest version.

Please note that Neato does not suply a USB cable so you need to get a mini USB cable when you want to update the software (firmware) of the robot. Bad news for Apple and Linux users: the firmware update software only runs on Windows. You can update the robot from Windows running inside VMware (Workstation or Fusion).

Take a look at Neato's update page to see if new updates are available.

Hacking the XV-15

When the robot is connected to the computer through USB, you can communicate with the device through Hyperterminal or Minicom. If you like hacking your robot, continue reading here.

Conclusion

I'm quite happy with the robot. The biggest question is how long this device will last. At first, a robot like this seems a bit as a toy and it may be, but it is a pretty darn useful one.

The lack of maintenance compared to the other robots is a big plus to me. If you have to spend time on cleaning the robot itself, where is the benefit?

To me, the only downside is the noise.

It can't vacuum the stairs. It can't vacuum in every corner. But the device can clean the majority of your house more often than you would probably have done yourself.

Additional sources

Robot Reviews

Very cool youtube film showing the robot through 'near infrared' view.

Lion's FileVault does not support Bootcamp and external boot disks

2011-08-05T01:00:00+02:00

Read the comments as they may provide useful information for your particular situation

I boot my iMac from an external FW800 SSD. I found out that it is impossible to encrypt this disk using the new FileVault as part of Lion.

Furthermore, I also found out that if you have a disk with a Bootcamp partition FileVault will also refuse to start the encryption process. I'm not trying to encrypt the Bootcamp volume, just the bootable Mac OS X Lion installation.

It may be advised to stay away from Lion if you need a setup similar to this one and also need disk encryption.

Lion's Disk Utility not compatible with CoreStorage and Filevault

2011-08-03T00:00:00+02:00

My 1 TB hard drive of my 2011 27" iMac was partitioned with:

A bootable partition with Mac OS X Lion
Time machine partition (I use an external SSD as my main OS)
Bootcamp partition with windows
A data partition containg... data.

The problem is that CoreStorage is too new. Disk Utility in Lion cannot cope with CoreStorage volumes. So when I decided to encrypt the bootable partition using the new Full Disk Encryption based on filevault, I could no longer manage my other partitions.

Furthermore, bootcamp got killed since it needs to be installed on one of the first three partitions on the disk. Due to the whole CoreStorage stuff and filevault, it became the fifth partition and it got killed. I couldn't get it back to life It wouldn't boot.

What I want now is to create a setup where I have three partitions:

A bootable (Boot) clone of my external FW800 SSD boot disk using SuperDuper
A Bootcamp volume running Windows (for games)
A data partition storing well.. data.

I want to encrypt the boot disk and the data partition. If this is going to work, I don't know.

It may be advised to stay away from Lion if you need a setup similar to this one and also need disk encryption.

Additional proof that Apple is ditching the optical drive

2011-07-23T15:00:00+02:00

I'm a strong advocate of killing the optical drive. As of 2011, there is no need for it anymore. Laptops could get lighter, smaller or have more room for additional battery capacity if the optical drive would no longer be present.

In my life, I never see people use the optical drive. And why would you use them any more? Isn't it so that if you are still using CDs or DVDs with your computer, that you do it out of (a bad) habit? And if you really can't part with your CDs or DVDs, would an external USB optical drive be a usable solution?

I think that we are at a point where most people don't even know that their computer has an optical drive.

With the release of the new 2011 Mac Mini, Apple dropped the optical drive yet again. They first dropped it from the MacBook Air and now the Mini.

What is next? Well that is clear. New Macs will be able to boot over the Internet from Apple's servers. Again, no need for an optical drive, even for reinstalling your computer. I wouldn't be surprised if the next generation of MacBook Pro laptops would not contain an optical drive. Maybe some people aren't ready for it but people should rejoice since it would make MacBooks thinner and lighter.

As apple killed the floppy drive, it is now killing the optical drive.

Cannot access Windows guest within VMware Fusion when running vsphere client

2011-06-17T09:00:00+02:00

Currently, I am running VMware ESXi 4.1 on a test system. To manage ESXi, you need the VSphere client, which is only available for the Windows platform. Therefore, I run VMware Fusion on my Mac to be able to access VSphere and manage my ESXi host.

The trouble is that both ESXi and VMware Fusion use the control-alt shortcut to release a console. So as soon as you start using a console within the VSphere client which itself runs within VMware Fusion, you cannot get back to the Windows OS.

You will have either access to Mac OS X or the ESXi guest. And to top it of, the mouse just completely disappears on Windows.

To get arount this problem, you need to somehow be able to send a control-alt sequence to the Windows guest withouth actually pressing control-alt.

Fortunately, VMware fusion allows you to create a key mapping that allows this.

Within the preferance pane, the first tab is called Key Mappings. You can create a new key mapping. For example, I mapped control+q to control-alt. This allows me to get out of the ESXi guest within the VSphere client, witouth getting grown back to Mac OS X. As a side effect, the mouse also showed up again, which is to be expected.

Determining smartphone market share using wireless sniffing

2011-04-24T23:00:00+02:00

I started a project to see if I could track smartphone users by sniffing for wifi-clients. Most smartphones support wifi and most people don't bother disabling wifi when they go outdoors1. If wifi is left on, it is possible to detect these smartphones and track their movement. To be able to track smartphones, all I had to do is grab a computer with a wifi card and start to listen for nearby smartphones.

Over de course of 10 days I was able to detect around 590 unique wireless client devices passing the vicinity of my house (near Amsterdam, The Netherlands). Please note that not all of those devices are smartphones, so I have to determine which are and which are not. I just used airmon-ng to sniff for wifi clients.

It is very easy to track a person if wifi is enabled on their smartphone since the phone will broadcast its unique identifier: its MAC-address. A MAC-address is as unique as a phone number so ideal for tracking down people. A single wifi sniffing computer is not enough to follow people, but if you would setup a grid of wifi sniffing devices, tracking people would be very easy.

Then I got bored with this project and decided that if I could get any additional information out of my data set of 590 wifi clients. The fun thing is that the first tree parts of a MAC-address disclose the vendor of the device. For instance, this MAC-address (made anonymous) belongs to HTC, thus is probably an HTC smartphone.

90:27:E4:B7:XX:XX

There is a whole list that shows which MAC-addresses belong to which manufacturers. This allows me to create a list of vendors associated with the MAC-addresses I captured. This is fun, because I can now count how many devices I 'caught' from a particular vendor.

The majority of wireless devices are from Apple (64%). The second largest is HTC (12%). That is an incredible difference between number one and number two. If these numbers actually mean anything, they are very interesting.

I think this picture is telling, but it's accuracy can be questioned. There are some problems with my data set. For instance, maybe many people using a particular brand of smartphone who do often disable wifi to conserve battery life.

Also, look whose missing in this list: Sony Ericsson. Are Dutch people not using Sony Ericsson smartphones? I must say that I deliberately used a Sony Ericsson smartphone to test my setup and it detected the device without any problem. So Sony Ericsson devices might not be that popular.

The main question is which conclusions can be drawn from this data: that iOS users often leave their wifi enabled and more than other smartphone users?

It is difficult to say what this data actually means and how accurate it is, but it may be an interesting technique none the less for real-life sampling of a smartphone population.

1 Unless your phone is so crappy that it won't hold a charge through the day with wifi enabled.

Setting up a Jabber instant messaging server |_http-title: Site doesn't have a title (text/html; charset=utf-8).

2011-02-05T22:00:00+01:00

I wanted to see how dificult it is to setup an instant messaging server based on open source software. Now I know that it is very easy, unless you are stubborn and do things your own way. In this example, I'm setting up a small IM server that is only for internal company use, but there is no difference if you want to expose the server to the internet.

First a bit background information. There is an open IETF standard for instant messaging called "XMPP" which stands for "Extensible Messaging and Presence Protocol". This protocol originated as part of the open source Jabber IM server software.

Setting up ejabberd

I decided to use ejabberd which is part of the Debian software archive. It is written in Erlang, but I can live with that. This blog posts documents how to setup the IM server with two accounts that can chat with each other. The configuration I use also enforces the use of SSL/TLS so authentication and all messages are encrypted.

Steps to get things running:

apt-get update
apt-get install ejabberd
cd /etc/ejabberd
edit ejabberd.cfg

Change the following line to your needs:

%% Hostname
{hosts, ["localhost", "jabber.domain.local"]}.

Also enforce the use of encryption like this:

starttls, {certfile, "/etc/ejabberd/ejabberd.pem"}

Must be changed to:

starttls_required, {certfile, "/etc/ejabberd/server.pem"}

Generating a custom SSL certificate

Security wise, it is very wrong to use the default SSL certificate as provided by the installation package for the server certificate. Anyone with access to this key material can decrypt encrypted communication. So you must generate your own server certificate. This is also required because IM clients may verifiy the certificate against the domain name used within the certificate. If there is no match, it will not work or it will at least complain.

openssl req -new -x509 -newkey rsa:2048 -days 365 -keyout privkey.pem \ 
-out server.pem

So this creates a public key (server.pem) and a private key (privkey.pem) which are valid for a year. Feel free to make the certificate valid for a longer period, this is an example. You will have to fill in some stuff, the most important part is this part:

Common Name (eg, YOUR name) []:jabber.domain.local

You are forced to set a password on the private key, but we want to remove this because otherwise the ejabberd service will not start automatically.

openssl rsa -in privkey.pem -out privkey.pem

Just enter the password you entered earlier and you're done. We now have separate files for the public and private key, but ejabberd expects them in one single file.

cat privkey.pem >> server.pem
rm privkey.pem

Set proper file system permissions:

chown ejabberd server.pem
chmod 600 server.pem

Now we are done. Restart ejabberd to use the new settings.

/etc/init.d/ejabberd restart

Security caveats

Please note that the ejabberd daemon provides a small build-in web interface for administration purposes on TCP port 5280. By default it is not protected by SSL or TLS and cannot be used unless you add users to this part of the confiuration file:

{acl, admin, {user, "", "localhost"}}.

Example:

{acl, admin, {user, "admin", "localhost"}}.

The user must also be registered as a normal IM user as described in the next section.

Warning: it seems to me that this interface is not very secure, for example, there is no logout button.

Furthermore, you might consider disabling the following section:

ejabberd_s2s_in

This prevents your IM server from communicating with other IM servers source. But we are not finished. When you install ejabberd, some other services are also started on the system. It is thus very important that you configure your firewall to block these ports. This small nmap port scan output shows some interesting services:

4369/tcp  open  epmd?
5222/tcp  open  jabber  ejabberd (Protocol 1.0)
5269/tcp  open  jabber  ejabberd
5280/tcp  open  http    ejabberd http admin
|_http-methods: No Allow or Public header in OPTIONS response (status code 400)
36784/tcp open  unknown

Port 4369, 36784 and 5280 should be blocked by your firewall and not accessible from the internet.

Adding users

It is now time to create some IM users. A user account always looks like an email addres, for example:

peter@jabber.domain.local

To add accounts, use the ejabberdctl utiliy:

ejabberdctl register peter jabber.domain.local <password>

Please note that passwords that are entered on the command line end up in your bash_history file, so beware. Also, users running ps aux may be able to see the command for a brief moment. So be carefull.

By registering two account, you can test your new server.

Additional resources

Nice to know: the domain names used for your accounts can differ from the domain used for the IM server.

If you have a Windows Active Directory domain, you could consider authenticating your users against LDAP.

Other resources: - tutorial 1 - tutorial 2

'Improved' image gallery for Blogofile

2011-01-21T22:47:00+01:00

This blog is just static HTML pages that are generated using Blogofile. The Blogofile website sports a small image gallery that has been written to illustrate how to create your own 'controllers' or Blogofile plugins.

My Python skills are horrible buy I managed to improve a bit on this example photo gallery. If you click on the 'Photos' menu at the top of this blog, you can see an example of the result.

What is the improvement?

Basically one thing: it create a gallery by traversing recursively over a directory structure containing pictures.

How to install?

Just create a folder where you will store pictures.
Add this to your _config.py

photo_gallery = controllers.photo_gallery

photo_gallery.enabled = True

photo_gallery.path = "pictures"

put this file into _controllers/photo_gallery
enter into this gallery and create a symlink like:

ln -s photo_gallery.py __init__.py

download the example templates here.

Additional info

This photo gallery generator ignores symlinks. However, it looks for a special file name in every directory:

showcase.jpg

This file can be a symnlink to one of the photo's. It is used as the identifying picture for a folder with pictures. Every directory should contain at least a single showcase.jpg.

If you have a folder called Europe containing two sub folders England and France, you can create a showcase.jpg symlink to one of the underlying showcase files of one of the folders. This picture will then be used to 'identify' the "Europe" folder.

Example directory tree:

./Europe
./Europe/England
./Europe/England/London
./Europe/England/London/showcase.jpg
./Europe/England/showcase.jpg
./Europe/France
./Europe/France/Nice
./Europe/France/Nice/showcase.jpg
./Europe/France/showcase.jpg
./Europe/Netherlands
./Europe/Netherlands/Enkhuizen
./Europe/Netherlands/Enkhuizen/showcase.jpg
./Europe/Netherlands/showcase.jpg
./Europe/showcase.jpg
./USA
./USA/New York
./USA/New York/showcase.jpg
./USA/showcase.jpg

Migration from blogger to blogofile and disqus is complete

2011-01-04T00:58:00+01:00

As stated in my previous post, I migrated all posts from blogger.com to my own host running blogofile. At the time of my previous post, I was not able to restore all comments made on my old blog. Thanks to some help from disqcus I fixed an error on my behalf and all old comments from my older blog appeared on the appropriate pages.

This completes the transition from blogger to blogofile and discus (for comments).

I'm now working on a controller for blogofile that recursively generates a static HTML photo blog. Relearning Python along the way. When working properly, the code will be made available so other people can at least have a good laugh.

This blog is now running on an old Intel based Mac Mini, also acting as a firewall.

Migrated this blog from blogger.com to blogofile

2010-12-30T19:21:00+01:00

Until now, I was hosting my blog on Google's blogger. I switched to using Blogofile. I wanted to have more control over my content and the layout.

I had many issues with the blogger blog post editor, resulting in ugly posts with too much white space, strange fonts and fonts sizes. Also, showing programming code examples with syntax highlighting is not possible with Blogger. As a nerd, I'm a control freak and I was not in control enough over my blog on blogger.

So I first started thinking about hosting my own wordpress site, but soon I concluded that this is not what I want. Wordpress has similar limitations as blogger and I no longer want to be depending on some special blog software without direct access to my content. So after a tip of a friend of mine I started to look at blogofile. Blogofile generates a static HTML website, which is a real benefit if you are hosting your own site. The most important benefits are:

Security
Performance
Ownership

Security

Most websites get hacked because they are basically live applications accessible from the Internet. Applications can contain bugs. If it contains security related bugs, you are toast. Software like wordpress has an awful track record regarding security. Basically, if you are running a wordpress site, you must be in a state of constant fear. You also constantly have to upgrade to the latest version within a short time frame or you will be hacked. I think that wordpress is a security hazard.

Blogofile generates plain text static HTML web pages. There is no dynamic content and thus no program running, except for the web server itself. There is no serious security risk.

Performance

Plain text web sites are fast. It is just data. When a visitor requests some web pages, there is no application running, laying pressure on the CPU, just plain data transfers. I bet you could host this blog without much performance problems on a recent smart phone like an Android or iOS device.

Ownership

All content is stored in a text-only format that can always easily be transformed into some other format. I use markdown syntax to write new blog posts. But plain HTML can also be used. Comments can be posted through discus. It seems however that I will loose all existing comments of my original blog. New comments are showing up without a hitch, but existing imported comments do not show up. I do want to preserve them but how? I'm not sure yet. I'm thinking about incorporating them in my posts as an appendix.

The whole process of migrating to blogofile

All steps taken so far as to migrate away from blogger:

Create a wordpress installation
Export blogger blog.
Import blogger blog in wordpress installation
'Import' the converted wordpress blog to blogofile 'format' using wordpress2blogofile.py
Hack, script and fight to change all URLs so they are correct for the new site.
hack, script and kill to get the old comments to show up, to no avail.
Learn markdown
Learn blogofile
Create + steal a new website design and have to learn css and html all over again.
Convert all posts from html to markdown.
Fix all converted posts where necessary
Put all external hosted content into a local folder and fix all links.

'Linux: using disk labels to counter storage device name changes'

2010-11-22T07:08:00+01:00

My router decided to change the device name for some USB storage devices. So /dev/sdc was swapped for /dev/sdd and vice versa. The result was some file system corruption on /dev/sdc, because it was used on a remote system through iSCSI, using a different file system from /dev/sdd.

With regular internal disks, attached with PATA, SATA or SAS, the chances are very small that such an event will occur, but it is possible, especially if you start adding/subtracting disks. With USB devices the risk is substantially bigger.

To prevent your system from mixing up drives because there device names change, use file system labels. All information that follows have been stolen from this location. Since this blog is also my personal notepad, the relevant bits are reproduced here.

There are three steps involved, the third being optional:

add a label to the file system
add the label to /etc/fstab
update grub boot manager (optional)

Add a label to the file system

Setting a label when the file system is created:

mkfs.ext3 -L ROOT /dev/sda1
mkfs.xfs -L BIGRAID /dev/sde

Set label for existing file system

EXT3:

e2label /dev/sda1 PRIMARY_ROOT
e2label /dev/sda1

XFS:

xfs_admin -L DATA1 /dev/sdf
xfs_admin /dev/sdf

Set label for swap partition

mkswap -L SWAP0 /dev/sdb5

add the label to fstab

Example of contents of fstab:

LABEL=ROOT          /         ext3    defaults        1 1
LABEL=BOOT          /boot     ext3    defaults        1 2
LABEL=SWAP          swap      swap    defaults        0 0
LABEL=HOME          /home     ext3    nosuid,auto     1 2

Update the grub boot manager

title server
root (hd0,0)
  kernel (hd0,0)/vmlinuz ro root=LABEL=SERVER_ROOT0 rhgb quiet
  initrd (hd0,0)/initrd.img

'Secure programming: how to implement user account management'

2010-11-18T21:40:00+01:00

Most web applications work like this:

The application uses a single database account to perform all actions. Users are just some records in a table. Account privileges and roles are part of this table, or separate tables.

This implies that all security must be designed and build by the application developer. I think this is entirely wrong. There is a big risk:

In such applications, SQL-injection will allow full control of the entire database.

This is something that is often overlooked. And the solution is simple. The application should not use a general account with full privileges. The application should use the database account of the user accessing the application. All actions performed by this user are thus limited by the privileges of this database account. The impact of SQL-injection would be significantly reduced.

The public part of a website is still using an application account, but the privileges of this account can be significantly reduced. To obtain elevated privileges, a user must first authenticate against the application and thus the database.

Please understand another benefit: it is not required to store username/password combinations of privileged accounts on the application server. The configuration file will only contain the credentials of the unprivileged account. An attacker compromising the application server with limited privileges, won't have access to the database with elevated privileges.

I understand that this solution requires a bit more work to setup at the start, but once implemented, it reduces complexity and improves security so much.

Of course, the security of your data is as good as the hardening of your database server. But that's another story.

The iPhone, iPad and iOS are powering a revolution

2010-11-06T01:12:00+01:00

Most people just don't understand computers. Are these people dumb? Some may be dumb, but the people who make them are maybe even dumber. Because they can't seem to figure out how to create a computer that the majority of people understand.

When the original macintosh arrived at the stage back in the eighties, computers became a bit more human-friendly, but it was limited to the constraints of the then available hardware. It put away the text-based interface and introduced the graphic interface. It used the desktop metaphor to create this graphic environment. But this metaphor has had its day.

Many people don't understand the desktop metaphor since they don't have a desktop and have never used one. Also, it is a metaphor, it's to translate the computer environment to something humans understand. But what if they don't understand the metaphor? For example, many people just don't 'get' the Windows Explorer or the Mac OS X finder. The desktop metaphor does not seem to fit in how people think.

Every time you see a person enter a URL like www.youtube.com in the google search field, you will realize that we still have a long way to go.

Most people did not seem to realize back then that the release of the iPhone wasn't that important, but the release of iOS. The iPhone was the first smartphone (a word most people are not familiar with) that did away with a stylus or hardware keyboard. It uses what is closest to us: our fingers. A totally new user interface, one that is very natural and close to us, is now available.

Using touch as input required a total redesign of the entire user interface. All other interfaces were designed around hardware keyboard and mouse devices. Fingers are big, and are obstructing the view. But it allows for a more direct interaction with a device. And now all new smart phones sport a touch interface.

Rumors of an Apple tabled existed for long, but it was very clear when the iPhone was released that if Apple would release a tablet, it would run this new iOS operating system.

When the iPad was released, it became an instant hit. As of today, there is no device on the market that can be truly called a competitor. But why is this so? The ground work has been done by the iPhone. Most people with an iPhone will notice that aside from some performance issues in the past, the device just always worked. It was instantly available to sent an email, look something up on wikipedia or find the nearest Starbucks. An iPhone just always works. No boot. Very reliable. And an interface that makes you happy.

Why does iOS make people happy? Because it provides a user interface that is human. People understand it instinctively. Any person of any age or background will be able to use an iOS device within minutes. The interface doesn't make you look like you are dumb because you just don't understand how it works. It not only works, it is easy to use and you are not afraid to break anything.

The iPhone and the iPad are learning a lot of people not to fear computers.

The iOS does away with the old desktop metaphor, but so does Symbian or similar interfaces. It is the combination with touch and the well thought out interface that sets it apart from other mobile operating systems. Even when the iOS platform did not have native applications, people still bought it and not only because Apple released a new shiny toy.

However, the app store on iOS has created a very special and important environment. People can finally install and remove applications in an extremely simple way. They don't need to be scared that some program will crash your computer either while installing it, using it, or removing it. The whole iOS ecosystem creates an environment wherein people don't need any help any longer from other people. They are finally in control. They don't need to be afraid of their computer.

This trend will affect the old-school user interfaces such as Mac OS X. How it will turn out is anybodies guess. But there is at least a small trend to 'eradicate' the finder as much as possible. iPhoto stores your photos. iTunes stores your music. If you want to include a photo or song within an application, you pick the photo or song in question from a miniature iPhoto or iTunes interface. There is no finder anymore. The finder is disappearing from the workflow. And why not? If programs are written well, why bother with it? The finder should be abstracted away, as is the case on iOS, where you don't have a finder.

Another thing is multitasking, you know, that stuf we like to do, but cant. We can only do one thing at a time. What we do want is fast task switching, not multitasking. Sure, some programs must be running in de background, to continue to operate, such as a chat program, but that is not the point. Most people are just going crazy if you show how multitasking works, with different windows. Again, iOS shows how 'multitasking' should be implemented. It is implemented as fast application switching, allowing these applications to register services that must continue to run, while the application itself freezes when the user switches to another application. People tend to use one application at a time and especially on mobile devices, every single bit of screen real estate counts, so they are always running full screen. This full screen notion will also be incorporated in the next Mac OS X release, Lion. People switch, but do one thing at a time.

Computer nerds tend to feel superior to people who don't have much skill using a computer. This feeling of superiority is totally misplaced. They should be really humble. because up until the advent of iOS, nobody was able to create a human friendly computer interface. It is not the lack of understanding on the side of computer users, it is the lack of understanding on the part of the computer nerds on how normal humans think and act.

Simple, human friendly computer interfaces will liberate humanity from those pesky computer nerds. And that will cause a bit less suffering in the world I hope.

Apple is killing off the optical drive just like the floppy disk

2010-10-23T11:05:00+02:00

With the release of the new MacBook Air we are one step closer to killing off the cd-rom and the dvd. As with the previous MacBook Air, this device has no optical drive. And that is a good thing. People do not need an optical drive. You have the network and you have USB disks. They are faster, more reliable and have more capacity.

I may expect that in the upcoming years this trend may continue with the other laptops. Just as Apple killed the floppy disk, it is killing the optical drive, one step at a time. I'd rather have a smaller and lighter laptop or more disk or battery capacity, than an optical drive. So I hope this is a trend that will continue and all other manufacturers will follow.

'Zabbix Security: client-server communication seems insecure'

2010-09-27T22:35:00+02:00

Zabbix is a populair tool for monitoring servers, services and network equipment. For monitoring hosts, Zabbix provides an agent that can be installed on the hosts that must be monitored.

Based on the supplied documentation and some remarks on the internets, the 'security' of Zabbix agents seems to rely on an IP-filter. It only accepts traffic from a specific IP-address. However, the protocol that is used between the Zabbix server and agents is unencrypted and does not seem to employ any additional authentication.

With a man-in-the-middle attack, pretending to be the Zabbix server, you would be able to compromise all servers running Zabbix. If remote commands are enabled on these hosts, the damage that could be done may be something you don't want to think about. Or maybe you do. Although it is true that for such an attack to be possible, as an attacker you need access to a system within the same network (VLAN) as the server, but none the less, it is just not secure.

Personally I don't think that Zabbix is suitable for high-security environments, due to the lack of encryption of sensitive data and the weak authentication mechanism.

Zabbix should employ at least SSL as a means for encrypted transport and use a password or shared secret for authentication. Even better would be the use of client-side certificates such as implemented by the system management tool Puppet.

[update]

Please note that Nagios agents also seem to work this way, but I have no experience with Nagios so I can't say for sure.

And Nagios is widely deployed in the enterprise...

Solaris is an obsolete platform

2010-08-14T22:55:00+02:00

Assuming that the rumor is true and OpenSolaris will be slain by Oracle, we must conclude that the Solaris operating system is obsolete. Solaris can be considered legacy. Sun was a hardware shop and to sell their hardware, they needed a great operating system.

Sun had a great operating system. And the Solaris platform was popular for a long time. And I think that was for the right reasons, at that time. If you or your company is still running on a Solaris platform, it may be time to rethink this strategy.

I do not understand why Oracle bought Sun. Oracle sells software. Sun sells hardware. Sun had some great products, like Java, so I can see some reasons. In the past, Solaris and Oracle had a tight relationship. But the only thing Oracle may be doing right now is a vendor lock-in strategy, where you are totally dependent on both hardware and software from Oracle.

But people don't seem to buy this, literally. People do want to continue to run Solaris, because thats the platform the've invested in. But they don't want to pay for those exotic Solaris Sparc systems, often way more expensive than commodity x86 hardware.

Oracle invested bilions in Sun assets. How are they going to make money of it? Squeeze out existing Sun Solaris customers who are depending on their platform?

If you are setting up a new business or if you think you can pull this off: stay away from this legacy platform. Migrate away from Solaris. Use an open platform that does not lock you in.

And this is also a very interesting read.

The future of ZFS now that OpenSolaris is dead

2010-08-14T10:51:00+02:00

With the probable loss of OpenSolaris, there may be another, maybe more devastating loss.

The very popular and very advanced Zetabyte File System (ZFS)

The only open source platform that actively supports ZFS is FreeBSD. And they just 'copied' the code from OpenSolaris. Are they able to maintain and further develop ZFS on their own? I don't think the community can handle a thing like that. Development on ZFS will severely be hampered and will not continue in the pace it did.

It is also clear that Oracle doesn't give a shit about open source or open operating systems. That is ok with me, everyone is entitled to their own opinion. But keep this in mind when you decide to use any Oracle product whatsoever.

It's not that I'm suggesting that you should not buy Oracle stuff. I have no grudge against Oracle in any way, it is just an objective observation, just be aware of this issue.

From the perspective of Oracle: what is their benefit regarding OpenSolaris? I understand their decision, but its sad nevertheless. And I'm really scared for the future of ZFS.

'A Moment Of Silence: OpenSolaris Is Dead'

2010-08-04T15:39:00+02:00

The last release of OpenSolaris dates back to July 2009. The next release was scheduled for March 2010, but Oracle did not release anything. It is dead silent around OpenSolaris.

On 12 July 2010, the OpenSolaris Government Board sent out an ultimatum to Oracle: "please communicate with us or we will resign and hand over the little power we have back to Oracle".

These are all signs that OpenSolaris has no real future. And think about it: what is Oracle's interest in OpenSolaris? Maintaining an operating system and develop new features is extremely expensive. What is their benefit? I can't see any. Sun was a hardware manufacturer and promoting OpenSolaris made sure that people got experience with the operating system that powers their hardware.

Unless your environment already consists of Solaris-based systems, as of 2010, there is no reason to use Solaris any more. OpenSolaris was, until recent, the only operating system that supported ZFS natively, so the only choice if you really wanted to use ZFS. Now FreeBSD has native support for ZFS too.

For the people who are fanatical about ZFS this is great, because it would be a shame if ZFS could only be used in combination with a dead operating system that supports less hardware than Mac OS X.

But I see no future in OpenSolaris and you should think twice about running it, either as a production platform or at home. Use a platform that has a large user-base and a big community.

I think that if you are running OpenSolaris, you must start thinking about migrating to FreeBSD or Linux, however painful that may be.

Compiling Handbrake CLI on Debian Lenny

2010-08-03T19:31:00+02:00

In this post I will show you how to compile Handbrake for Debian Lenny. Please note that although the Handbrake GUI version does compile on Lenny, it crashes with a segmentation fault like this:

Gtk: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 17

(ghb:1053): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

(ghb:1053): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

(ghb:1053): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed

Segmentation fault

So this post only describes how to compile the command-line version of Handbrake: HandBrakeCLI.

Issue the following apt-get commando to install all required libraries and software:

apt-get install subversion yasm build-essential autoconf libtool zlib1g-dev libbz2-dev intltool libglib2.0-dev libpthread-stubs0-dev

Download the source code at http://sourceforge.net/projects/handbrake/files/
Extract the source code and cd into the new handbrake directory.
Compile handbrake like this:

./configure --disable-gtk --launch --force --launch-jobs=2

The --launch-jobs parameter determines how many parallel threads are used for compiling Handbrake, based on the number of CPU cores of your system. If you have a quad-core CPU you should set this value to 4.

The resulting binary is called HandBrakeCLI and can be found in the ./build directory. Issue a 'make install' to install this binary onto your system.

Debian Lenny and Dell S300 H200 and H700 RAID controllers

2010-07-10T21:50:00+02:00

update november 2010: it is reported in the comments that the next release will support these controllers. However, which controllers are supported is not clear. Also, we may have to wait for quite some time before squeeze will be released.

Just a quick note: it seems that the new Dell RAID controllers are not supported by the current stable version of Debian: Lenny. The S300, H200 and H700 controllers as supplied with the R310, R410 and 'higher' systems, are thus not a great choice if you want to keep things 'stock'. You may have to run backported kernels and installer images and I couldn't figure out if these controllers actually work with these latest images.

The Perc 6/i controller is supported by Debian. It seems that it can only be supplied with the R410 and higher systems. If you have any additional information, like actual experience with these controllers, please report this.

I installed VMware ESX 4.1 and installed DEbian Lenny om top of that without problems.

Please note that the network cards of the R410 are still not supported and will cause a headache!

-network-card.html

Secure caching DNS server on Linux with DJBDNS

2010-06-12T16:37:00+02:00

The most commonly used DNS server software is ISC BIND, the "Berkeley Internet Name Daemon". However, this software has a bad security track record and is in my opinion a pain to configure.

Mr. D.J. Bernstein developed "djbdns", which comes with a guarantee: if anyone finds a security vulnerability within djbdns, you will get one thousand dollars. This price has been claimed once. But djbdns has a far better track record than BIND.

Well, attaching your own name to your DNS implementation and tying a price to it if someone finds a vulnerability in it, does show some confidence. But there is more to it. D.J. Bernstein already pointed out some important security risks regarding DNS and made djbdns immune against them, even before it became a serious world-wide security issue. However, djbdns is to this day vulnerable to a variant of this type of attack and the dbndns package is as of 2010 still not patched. Although the risk is small, you must be aware of this. I still think that djbdns is less of a security risk, especially regarding buffer overflows, but it is up to you to decide which risk you want to take.

The nice thing about djbdns is that it consists of several separate programs, that each perform a dedicated task. This is in stark contrast with BIND, which is one single program that performs all DNS functionality. One can argue that djbdns is far more simpler and easy to use.

So this post is about setting up djbdns on a Debian Linux host as a forwarding server, thus a 'DNS cache'. This is often used to speed up DNS queries. Clients do not have to connect to the DNS server of your ISP but can use your local DNS server. This server will also cache the results of queries, so it will reduce the number of DNS queries that will be sent out to your ISP DNS server or the Internet.

Debian Lenny has a patched version of djbdns in its repository. The applied patch adds IPV6 support to djbdns. This is how you install it:

apt-get install dbndns

The dbndns package is actually a fork of the original djbdns software. Now the program we need to configure is called 'dnscache', which only does one thing: performing recursive DNS queries. This is exactly what we want.

To keep things secure, the djbdns software must not be run with superuser (root) privileges, so two accounts must be made: one for the service, and one for logging.

groupadd dnscache

useradd -g dnscache dnscache

useradd -g dnscache dnscachelog

The next step is to configure the dnscache software like this:

dnscache-conf dnscache dnscachelog /etc/dnscache 192.168.0.10

The first two options tell dnscache which system user accounts to use for this service. The /etc/dnscache directory stores the dnscache configuration. The last option specifies which IP address to listen on. If you don't specify an IP address, localhost (127.0.0.1) is used. If you want to run a forwarding DNS server for your local network, you need to make dnscache listen on the IP address on your local network, as in the example.

Djbdns relies on daemontools and in order to be started by daemontools we need to perform one last step:

ln -s /etc/dnscace /etc/service/

Within a couple of seconds, the dnscache software will be started by the daemontools software. You can check it out like this:

svstat /etc/service/dnscache

A positive result will look like this:

/etc/service/dnscache: up (pid 6560) 159 seconds

However, the cache cannot be used just yet. Dnscache is governed by some text- based configuration files in the /etc/dnscache directory. For example, the ./env/IP file contains the IP address that we configured previously on which the service will listen.

By default, only localhost will be able to access the dnscache. To allow access to all clients on the local network you have to create a file with the name of the network in ./root/ip/. If your network is 192.168.0.0/24 (thus 254 hosts), create a file named 192.168.0:

Mini:/etc/dnscache/root/ip# pwd

/etc/dnscache/root/ip

Mini:/etc/dnscache/root/ip# ls

192.168.0

Now clients will be able to use the dnscache. Now you are running a simple forwarding DNS server and it probably took you under ten minutes to configure it. Although djbdns is not very well maintained in Debian Lenny, there is currently not a really good alternative for BIND. PowerDNS is not very secure (buffer overflows) and djbdns / dbndns has in more than 10 years never been affected by this type of vulnerability.

RAID array size and rebuild speed

2010-04-03T23:02:00+02:00

When a disk fails of a RAID 5 array, you are no longer protected against (another) disk failure and thus data loss. During this rebuild time, you are vulnerable. The longer it takes to rebuild your array, the longer you are vulnerable. Especially during a disk-intensive period, because the array must be reconstructed.

When one disk fails of a RAID 6 array, you are still protected against data loss because the array can take a second disk failure. So RAID 6 is almost always the better choice. Especially with large disks (1+ TB), because the rebuild time largely depends on the size of a single disk, not on the size of the entire RAID array.

However, there is one catch. The size of the RAID array matters when it becomes big, 10+ drives or more. No matter if you use hardware- or software- based RAID, the processor must read the contents of all drives simultaneously and use that information to rebuild the replaced drive. When creating a large RAID array, such as in my storage array, with 20 disks, the check and rebuild of the array becomes CPU-bound.

This is because the CPU must process 1,1 GB/s (as in gigabyte!) of data and use that data stream to rebuild that single drive. Using 1 TB drives, it checks or rebuilds the array at about 50 MB/s, which is less than half what the drives are capable of (100+ MB/s). Top shows that indeed the CPU is almost saturated (95%). Please note that a check or rebuild of my storage server takes about 5 hours currently, but that could be way shorter if the CPU was not saturated.

My array is not for professional use and fast rebuild times are not that of an issue. But if you're more serious about your setup, it may be advised to create more smaller RAID vollumes and glue them together using LVM or some similar solution.

'Linux: monitor a directory for files'

2010-03-22T17:01:00+01:00

Inotify is a mechanism in the Linux kernel that reports when a file system event occurs.

The inotifywait comand line utility can be used in shell scripts to monitor directories for new files. It can also be used to monitor files for changes. Inotifywait must be installed and is often not part of the base installation of your Linux distro. However, if you need to monitor a directory or some task like that, it is worth the effort. (apt-get install inotify-tools).

Here is how you use it:

inotifywait -m -r -e close_write /tmp/ | while read LINE; do echo $LINE | awk '{ print $1 $3 }'; done

Let's dissect this example one part at a time. The most interesting part is this:

inotifywait -m -r -e close_write /tmp/

What happens here? First, inotifywait monitors the /tmp directory. The monitoring mode is specified with the -m option, otherwise inotifywait would exit after the first event. The -r option specifies recursion, beware of large directory trees. The -e option is the most important part. You only want to be notified of new files if they are complete. So only after a close_write event should your script be notified of an event. A 'create' event for example, should not cause your script to perform any action, because the file would not be ready yet.

The remaining part of the example is just to get output like this:

/tmp/test1234/blablaf

/tmp/test123

/tmp/random.bin

This output can be used to use as an argument to other scripts or functions, in order to perform some kind of action on this file.

This mechanism is specific to Linux. So it is not a OS independent solution.

Scanning many hosts in parallel with Nmap using PPSS

2010-02-18T21:41:00+01:00

Scanning a large number of hosts using Nmap often takes a lot of time. During this time, no output is written to a file or disk. Only when Nmap is finished, is all output written to the output file. Often, I want to start processing results of hosts that have already been scanned. Often, the trick is to split the input file with all the hosts and start multiple Nmap instances by hand using the different input files. This is rather cumbersome. Now what I really want is that I get the results of a scan of a particular host immediately available as soon as it's finished. Here is where PPSS comes in. PPSS can start Nmap scans and proces a list of hosts as contained in an input file. PPSS will only start a predefined max number of simultaneous scans in parallel, as not to overwhelm the scanner, network or target hosts. This is an example on how PPSS can be used to obtain results immediately:

./ppss -f hosts.txt -c 'nmap -n -v -sS -A -p- -oN "$ITEM" "$ITEM"' -p 4

Where hosts.txt contains IP-addresses, networks or domain names like:

192.168.0.1

192.168.0.2

192.168.0.3

192.168.1.1-254

www.google.nl

The 'ITEM' part is the fun bit. In this example, multiple instances of Nmap will scan a single hosts. The output is written to a file called "$ITEM", which is of course substituted for the IP-address or domain name as read from hosts.txt. The second "$ITEM" is the argument to Nmap which tells which host to scan. The -p 4 option tells PPSS to run 4 nmap scans simultaneously at all times.

You will end up with a large number of output files, one per host. As soon as a scan is finished on one host, you can start processing the results, instead of waiting for that big scan to finish.

Corsair CM PSU-750HX seems ok

2010-02-10T23:59:00+01:00

I had to replace my Coolermaster PSU and after some searching on the interweb, I chose the Corsair CMPSU-750HX. One of the reasons is that Corsair states that this PSU can withstand 50 degree Celcius in continuous operation on full load.

The package is very, clean, with all the cables in some neat pouch and the PSU itself also in a neat bag. You pay quite some money, but it at least suggest quality. The modular design with the modular kables is excellent.

Personally, I think that PSUs with multiple 12v rails are not that usefull. This particular PSU has just one single 12v rail that can be loaded up to full capacity of the PSU. I think this makes the design easier and if you are running heavy systems that have short spikes (starting al drives) this is ideal.

I hope this one lasts longer.

Smoking coolermaster Silent Pro M 600W TN6M50

2010-02-10T00:46:00+01:00

So I was just messing around on my work station, when suddenly I smelled the smell any person familiar with electronics fears. The smell of some electrical component burning.

The whole upper floor smelled like something was smoldering. So I shut down both my storage servers. After half an hour I still hadn't found the source of the smell, but it definitely was my 18 TB NAS.

So I switched it back on to see if I could determine the cause. The smell became stronger and within the strike light of my flash light, I saw small strands of grey smoke slowly twirling out of the Coolermaster Silent Pro M 600W power supply.

That was the moment I decided to turn the system off immediately. I know this kind of thing can happen. But it is still very bad for such a component to crap out on me like that.

Needless to say that I don't want a replacement of this model in my server. I will now try my luck with the Corsair CMPSU-750HX 750Watt PSU.

Based on a single experience it cannot be concluded that this Coolermaster PSU is a bad one that should be avoided. But the only moving part of a PSU is the (big) fan. What must be wrong with it that it decided to start smoking?

For reasons of availability, a redundant power supply should be used, but that is just too expensive for my taste ($500+).

Is the iPhone OS a threat to our freedom?

2010-01-31T19:55:00+01:00

There is one fundamental flaw with both the iPhone and the iPad. As a user, you do not have full control over your device. You can only install or run software that is approved by Apple.

This is something that is unprecedented. All major platforms, Windows, Linux, Mac OS X do provide the user with full control over what programs can be run or not.

Is this a bad thing? Are you trading your freedom for some luxurious gadget such as the iPhone or the iPad?

I say no. People are over reacting.

You would trade in your freedom and become 'owned' by Apple, if an iPhone or iPad is your sole computing device.

However, everybody also has a computer at home, that is still under your full control.Since the iPhone and iPad is just an auxiliary device (although quite powerful), you are still free.

Most importantly, I think it is about the content, not the device. Do you retain control over your own content, such as music, files and e-mail? I think so. I think that on this level, which is most crucial, Apple does not restrict people on iPhones or iPads in any way. And that is what matters most I think.

Why the iPad will be a breakthrough in human computing

2010-01-30T10:50:00+01:00

The fact is that computers aren't made for humans. Computers are just made, and humans have to adjust to them.

The problem is that most people that are not into technology just don't understand how computers work. Should I single click of double click? Click left or right?

The only people that can work with some ease on computers are people with a more than average interest in technology. Ordinary people often have to fight great battles to get the job done. The therm 'ordinary' does not justice to these people, because technologists seem to forget that they are the people that actually do something useful.

The iPad is the first personal computer that is actually made for humans that do not have a primary interest in computers. The entire computer is abstracted away to focus on the things you really want to do. People do not need the help of technologists anymore to do the things they want.

The iPhone was the first in this new kind of computers, but due to it's size it's still a bit of a gimmick. But it was a revolution within the world of smart phones. It was the first device that was actually usable for non- technologists. And the iPad will be the first 'regular' computer that (hopefully) won't be such a nuisance as current home computers are.

The iPad will be the death of Flash

2010-01-30T10:16:00+01:00

So Apple finally released their tablet computer: the iPad. One of the most debated drawbacks is that it lacks support for Adobe Flash. The iPhone does not support Flash either, and since the iPad is based on the iPhone OS, this should not come as a surprise.

Now many people see this as a serious problem. There are a slew of websites that cannot be viewed on an iPad for this reason.

However, the thing is this: Flash sucks. It is a proprietary technology under the control of Adobe and the only reason it is currently popular is that it provides a kind of standard for video playback across all browsers and operating systems. But it still sucks. It crashes. It is shit.

The iPhone is immensely popular, despite the lack of flash. The iPhone user base is huge. By using Flash in your website, you are excluding a large population of potential visitors. So this is what will happen: websites will start dumping Flash. The iPhone and the iPad will together kill Flash.

And that is a good thing for everybody, especially for proponents of open standards and open formats. Oh irony, that it will take a closed proprietary platform to do so.

provides-the-ultimate

Mounting a file system or partition from a disk image

2010-01-23T10:13:00+01:00

You cannot just make a disk copy with dd and then just mount it as a regular disk. You must know where the partition starts on the disk. So first, you need to get the partition table with sfdisk:

sfdisk -l -uS image_file.dd

The output is something like:

Disk /mnt/image/image_file.dd: 9729 cylinders, 255 heads, 63 sectors/track
Warning: The partition table looks like it was made
for C/H/S=*/240/63 (instead of 9729/255/63).

For this listing I'll assume that geometry.

Units = sectors of 512 bytes, counting from 0

Device Boot Start End #sectors Id System

/mnt/image/simon-besmet.img1 63 8497439 8497377 b W95 FAT32
/mnt/image/simon-besmet.img2 * 8497440 156280319 147782880 7 HPFS/NTFS
/mnt/image/simon-besmet.img3 0 - 0 0 Empty
/mnt/image/simon-besmet.img4 0 - 0 0 Empty

Next, we need to calculate the actual starting point of the partition. The sector number is in sectors, which contain 512 bytes. So in this case, the starting point of the NTFS partition is 8497440 x 512 = 4350689280.

To mount the image, enter the following command, using the calculated offset:

mount -o loop,offset=4350689280 /mnt/image/disk_image.dd /mnt/disk

Source:

http://lists.samba.org/archive/linux/2005-April/013444.html

'Linux: show graphical layout of disk temperatures'

2010-01-03T03:12:00+01:00

graphic, representation

To get a visual representation of hard drive temperatures, I wrote a small script. The output of this script looks like this:

This output is tailored to the exact disk lay-out of my storage server. However, it is also usable for other servers. You have to edit the lay-out depending on the system.

This script assumes that you have smartmontools installed on your system. It uses smartctl to obtain the drive temperatures.

The script can be found here or here in tgz format.

Recovering a lost partition using gpart

2009-12-22T19:12:00+01:00

Even today people do not understand how important it is for the safety of your data to make backups. I was asked to perform some data recovery on a hard drive of an old computer, which still contained important documents and photo's.

The first thing I did was to make a disk image with ddrescue. I always work with the image and not with the original drive, to prevent any risk of accidentally messing things up for good.

Example:

ddrescue -r 2 -v if=/dev/sdf of=/storage/image/diskofperson.dd bs=1M

Next, I tried using gparted on this file but got this error:

Welcome to GNU Parted! Type 'help' to view a list of commands.

(parted) p

Error: /storage/image/diskofperson.dd: unrecognised disk label

(parted) quit

Also fdisk -l didn't work:

Disk /storage/image/diskofperson.dd doesn't contain a valid partition table

It seemed that the partition table was gone. I used the utility testdisk to recover this partition, to no avail. Why this tool didn't work is beyond me.

I found a very old utility called 'gpart' that just searches a disk for existing partitions. I just want to know the starting offset of the relevant partition.

So I ran:

gpart -g /storage/image/diskofperson.dd

And I got nothing useful, although a partition was found:

Begin scan...

Possible partition(DOS FAT), size(57255mb), offset(0mb)

End scan.

So I ran the command again with more verbosity:

gpart -g /storage/image/diskofperson.dd

...

Begin scan...

Possible partition(DOS FAT), size(57255mb), offset(0mb)

type: 011(0x0B)(DOS or Windows 95 with 32 bit FAT)

size: 57255mb #s(117258372) s(63-117258434)

chs: (1023/255/0)-(1023/255/0)d (0/0/0)-(0/0/0)r

hex: 00 FF C0 FF 0B FF C0 FF 3F 00 00 00 84 38 FD 06

End scan.

...

This time I got something useful. The s(63-117258434) part shows the starting sector, which is 63. A sector is 512 bytes, so the exact starting offset of the partition is 32256.

So to mount this partition, just issue:

mount -o loop,ro,offset=32256 /storage/image/diskofperson.dd /mnt/recovery

And voilá, access to the filesystem has been obtained.

/storage/image/jdiskofperson.dd on /mnt/recovery type vfat (ro,loop=/dev/loop0,offset=32256)

Wake On Lan not working with realtek r8168 card

2009-12-20T15:18:00+01:00

After messing around with different kernels on Debian Lenny, I noticed that my system did no longer respond to WOL packets. The system is using an on-board Realtek R8168 chip which supports wake on lan.

After searching and reading i found the problem. The system is using an r8168 chip, but the driver that is loaded by Linux is the r8169 driver. If the r8168 driver is loaded, as it should be, then WOL is working again.

Shunit2, unit testing for shell scripts

2009-12-17T16:47:00+01:00

This may be of interest to people who are as stupid as I am and write elaborate shell scripts instead of using a proper scripting language such as Python or Ruby. No I am deliberately not mentioning Perl here.

Anyway, testing is always an issue. With PPSS, I encountered many times that some change broke something in another place. Testing if some change screwed things up manually is just tedious and a pain.

As any person who is semi-serious about programming should have done already, I started to write some unit tests. For bash scripts, there is this unit test framework called Sunit2. It takes some effort to write tests. But once you have written some tests and witness how the tests that passed before now fail after you changed some code is priceless. Because often, you would not have found the issue before a long time.

Google DNS - what to think of it?

2009-12-03T18:42:00+01:00

Google now provide an open DNS service. At first I was scared that they use their service to get information about users.

However, their privacy statement tells us that no personal identifiable information is stored for longer than 48 hours. The permanent logs do not contain such information. The most notable information that is stored permanently is the requested domain. The other thing is that they record the location of the request on the city level. And even that info is only stored for two weeks. After that, only a small random sample of info is stored for permanent storage.

The main question is now: who do you trust more: Google or your ISP.

To be honest, I think I trust Google more than my ISP. I don't think my ISP adheres to the same privacy policy.

Convert dos file to linux format wit vim

2009-12-03T17:12:00+01:00

If you have some file that has been saved on a (Win)do(w)s host you may notice that all lines end with ^M.

To fix this, you can use the tool dos2unix. If however this tool is not at your disposal, you may have a problem. If vim is available, you do not have a problem:

The trick is to delete all occurances of ^M with this line:

:%s/ctrl-v ctrl-m//g

This will look like:

:%s/^M//g

Just save the file and you're done.

Which file system for a large storage array under Linux?

2009-11-30T13:03:00+01:00

There are many file systems available under Linux, however only a few of them can be used for a large storage array.

I am assuming that you want to create a single file system. I don't care if you use LVM or other layers beneath, this is about which file system to use.

I will discuss each file system in short.

EXT3

Old and trusted. but slow, wastes tremendous amounts of free space and has an 16 TB limit. Not recommended. At 16 TB, you will lose significant amounts of free space.

Developed by IBM, it seems an ok file system and it supports 16+ TB file systems. However it is abandoned by IBM and does not seem to have a future. The reason I did not choose JFS is because it does not seem to be in wide spread use and therefore, and I do not want to take any risk in this regard. I might encounter some rare bug because I would be one of the few Linux jfs users that creates 16+ TB file systems.

A file systems that works. Supports 16+ TB file systems and is widely used. It has some drawbacks, such as data loss (files are zeroed) when something goes wrong (powerloss) however, nobody that is interested in creating a large filesystem is going to run the box without a UPS.

Reiserfs

The guy who designed and wrote it is in jail for killing his wife. Not much future here, i suppose. Also, it does not support file systems bigger than 16 TB.

Reiser4

Some new version of Reiserfs which is still not available as part of a stable Linux kernel.

EXT4

Ext3 upgraded to current days need, although with a draw back. On paper, it supports 16 TB+

file system, but currently, in practice, it does not. I found out the hard way and had to reformat my array with another file system.

My choice:

Honestly, there is not much of an option currently under Linux other than XFS. In the past, I had some quirks with file names that contained strange characters causing trouble. However, I never lost any data whatsoever.

Currently, what I'm really hoping for is ZFS for Linux or a stable version of btrfs. These are truly modern day file systems with support for snapshots, etc. But this is still a dream. Until then, I will stick with XFS.

'Tip of the day: Scrolling in GNU screen'

2009-11-30T11:44:00+01:00

Just the answer:

By default, on Debian, the scroll back buffer is about 1K lines. This can be changed in the .screenrc file in your home directory. The following example sets the scroll back buffer to 5K lines.

defscrollback 5000

Set the scroll back buffer on the fly with:

First enter the command line mode:

ctrl + a, :

Then enter:

scrollback 2000

To set the scroll back buffer to 2000 lines.

To actually scroll back, the actuall stuff why you may be reading this post:

First enter copy mode:

ctrl + a, [

Use standard VI controls to navigate through the lines (h j k l).

Use ctrl+b to scroll a full page up

Use ctrl+f to scroll a full page down.

For more details, please visit this link:

scrollback-buffer/

'Gzip with parallel compression support: pigz'

2009-11-22T18:29:00+01:00

The speed at which files are compressed with gzip is currently almost always determined by the speed of the CPU. However, standard unix gzip is single- threaded and only uses a single CPU (core).

However, the maintainer of the zlib library has released 'pigz' or 'pig-zee' whichs adds just that: support for parallel compression. This dramatically improves the speed at which a file can be gzipped.

In this example, a 3 GB compressable file is gzipped:

Gzip:

root@Core7i:~# time gzip pigz.bin

real 1m58.994s

user 1m56.480s

sys 0m1.820s

Pigz:

root@Core7i:~# time pigz pigz.bin

real 0m31.524s

user 2m54.890s

sys 0m2.900s

This simple and a bit unscientific example shows a 400% speed improvement. Since the Core i7 has four real cores, this shows that pigz scales nicely.

How to determine which process causes IO ?

2009-11-22T15:19:00+01:00

There is a nifty little program called 'iotop'. Iotop is part of Debian or Ubuntu and can be installed with a simple apt-get.

Once you have determined with 'top' that the system is waiting on IO-access, It is nice to know

which process is responsibe for this IO. Therefore, you want a list of processes, just like top provides, but instead of CPU or RAM usage, it shows the IO a process is generating. This is exactly what 'iotop' provides.

This is ideal when troubleshooting performance problems caused by heavy IO.

http://guichaz.free.fr/iotop/

Making cowpatty recognize a four-way handshake

2009-11-22T12:52:00+01:00

I was unable to get cowpatty working with a packet capture that actually

contains a four-way handshake of a WPA session.

I got it working like this:

First, download cowpatty 4.6 right here, within the source directory of cowpatty.

Extract cowpatty and apply this patch using these instructions

Then build couwpatty just with 'make' and 'make install'.

I created a test setup with a known password. However, this patched

version did not find the passphrase using a dictionary file.

I then used genpmk to create a precomputed hash database like this:

genpmk -f ./aircrack-ng-1.0/test/password.lst -d hashes -s default

Please note that I added the correct passphrase to this password list to

make sure that cowpatty works.

Finally, using the -d option on the hash file, cowpatty

managed to crack the PSK.

ng.org/index.php?PHPSESSID=9e79f0ff83630bbe6c870b1a9aeef63c&topic=5867.0

Monitor power usage with your UPS

2009-11-18T21:09:00+01:00

If a system is connected to a UPS (Uninterruptible Power Supply), it is possible to determine how much power it consumes. For this purpose, I wrote a small script:

 Host:~# ./ups.sh

 UPS model: Back-UPS RS 1200 LCD
 APC model: Back-UPS RS 1200 LC
 Capacity: 720 Watt
 Load: 18 Percent
 Usage: 129 Watt
 Time left: 33 Minutes
 Status: ONLINE

 Host:~# ./ups.sh

 UPS model: Back-UPS RS 1200 LCD
 APC model: Back-UPS RS 1200 LC
 Capacity: 720 Watt
 Load: 19 Percent
 Usage: 136 Watt
 Time left: 22 Minutes
 Status: ONBATT

This script assumes that:

You are running a unix
You run apcupsd

The script can be downloaded here:

24 TB based on Norco RPC-4020 and Linux Software RAID

2009-11-14T13:07:00+01:00

Just a quick link:

Some person build basically the same setup, including identical controller, providing 28 TB of storage: Take a look here

The main difference is that this person uses 1.5 TB disk, thus achieving more storage.

Debian Linux Preseeding - an installation framework

2009-11-11T21:11:00+01:00

Look Ma!

No hands!

just watching a system PXE-boot and install itself into a fully operational system is fun. PXE boot and preseeding is not enough however to accomplish this.

By itself, preseeding only installs a basic operating system. Next, you need to configure all sorts of things and services, install additional software, create accounts. etc.

So there is a whole post-install process that you also need to automate. Preseeding by itself, as I understand it, does not seem to be able to do that for you. But you can execute commands at the end of the installation.

First, I'd like to point out that the alternative to Debian preseeding is, in my opinion, FAI or Fully Automated Installation. I want to make clear that this is a product with a long history that may better suit your needs. I do not use FAI because I find it way to complicated. It seems very powerful to me, but I don't understand it.

I understand preseeding and shell scripts. Putting commands that I would otherwise execute manually are now performed by some shell script. And I think every administrator understands shell scripts and will be able to automate installation with almost no effort by just executing a bunch of shell scripts that install various additional components.

I wrote a small installation 'framework' that can be used in conjunction with preseeding to truly fully automate your installation, without the requirement of learning complex stuff like FAI.

Unfortunately, it is not available yet on-line, but as soon as I find the time, a new project will be started and this small installation framework will be available with some example preseed configuration.

In day-to-day operations I use this script to install servers and laptops. Once you see it you will like it.

Linux Mac Mini - temperature monitoring with lm-sensors

2009-11-09T01:21:00+01:00

This post is about getting temperature monitoring to work with a Mac Mini running Linux.

Using Debian Lenny, out of the box, lm-sensors is not working. No sensors can be found. This is how temperature monitoring and fan speed monitoring can be made to work:

modprobe applesmc

If you run "sensors-detect" after this, and do a:

modprobe coretemp

Then "sensors" will give you ouput like this:

Mini:/sys/devices/platform# sensors

applesmc-isa-0300

Adapter: ISA adapter

Master : 2151 RPM (min = 1500 RPM)

temp1: +83.2°C

temp2: +68.0°C

Some software to control the fan speed:

http://stargate.solsys.org/mod.php?mod=faq&op=extlist&topicid=27&expand=yes#1 18

d=yes#118

Linux on Mac Mini - boot after power failure

2009-11-08T20:57:00+01:00

When using a Mac Mini as a server or router, it is very nice if the machine automatically boots if a power failure has occurred.

User chirhoxi on the ubuntu forum found out how this can be achieved:

http://ubuntuforums.org/showthread.php?t=1209576

Basically you need one of these commands:

Original Mac Mini:

setpci -s 00:03.0 0xa4.b=0

Newer Mac Mini:

setpci -s 00:03.0 0x7b.b=19

These commands might not work with the latest Mac Minis but the thread discusses how to determine for yourself what the appropriate command is.

How to run Debian Linux on an Intel based Mac Mini

2009-11-01T23:22:00+01:00

The Mac Mini is just a gorgeous device. It is beautiful, small, silent, powerfull yet energy efficient. When idle, it uses around 20 watts. I'm using one of the first Intel-based Minis with an Intel Core Duo chip, running at 1.6 Ghz.

I want to use this mini as an expensive router and download host. I could have used something embedded, such as one of those router boxes that costs about 70 euros, but no, I want to do some more with my router, such as downloading, etc. It is the only device in my house that is allowed to run 24/7 so it has to be a bit more powerful if I want more than just routing. I know that this mini was like 600 euros or something back in the days, and that is quite some money to spend on something that is now only a router. However, when I was still running Mac OS X on it, I didn't do much more with it than I will now, it will actually do more.

I am assuming that you want to run Linux exclusively on the Mac and that Mac OS X will be wiped off.

To get this puppy running Debian Linux (Lenny), you need to first boot the Mac with the (Snow) Leopard OS X boot CD and startup the diskutilily.

You need to create at least two partitions: one for the root file system and one for swap. The most important step is to select 'options' under the partition layout screen, and select Master Boot Record partitioning instead of the other 2 options. Do NOT use GUID or Apple Partition Map.

Now, boot your regular Debian Linux boot CD, I use the regular network installation CD. When you get to the partitioning screen, do NOT auto- partition the hard disk. Just reconfigure the existing partitions you just made using Diskutility. So the large partition will be configured as "/" and made bootable. The small partition must be configured as swap.

After the installation finishes, just install GRUB in the MBR and reboot. If all went alright, you will see a non-blinking folder on a gray background for a couple of seconds, after which Linux will boot. If you get a blinking gray folder with a question mark, something went wrong.

It seems that if configured properly, after the EFI boot mechanism fails to find a system folder on some Mac partition, the legacy BIOS emulation seems to kick in, and star to search for something to boot.

The Mini has only one network card, so another one is necessary to run it as a router. I bought some no brand USB2 to 100 MBIT NIC (Bus 005 Device 003: ID 9710:7830 MosChip Semiconductor MCS7830 Ethernet) which seems to run smoothly.

card with a dual gigabit card.

I guess you will need to mod the Mini but it will allow true gigabit speeds on all interfaces.

PCI_Express_Dual_Gigabit_LAN_Module/product_info.html

PPSS version 2.30 now operates asynchronous

2009-09-26T19:32:00+02:00

If you background a bash or shell process, how do you determine if it has finished? Since inter process communication is not possible using shell scripts, people often refer to while loops or other polling mechanisms to determine if some process has stopped.

However, the one player that knows best if a process has finished is the process itself. So if only this process could tell the parent or other process about this...

The solution is using a FIFO or 'pipe'. A listener process reads the pipe and executes a command for every message received through this pipe. This was already build-in into PPSS. However, PPSS had this dirty while loop that polls every x seconds to determine if there are still running workers. If not, PPSS finishes itself.

However, while loops and polling mechanisms are evil, dirty and bad. The nicest solution is to make PPSS fully asynchronous. To achieve this, every job must tell PPSS that is has finished. PPSS already has this listening process that listens to the pipe for commands. If this channel is used by workers to communicate that a worker is finished, PPSS will know when all workers are finished.

This makes PPSS a lot faster and responsive.

How to build an energy efficient computer for home use

2009-09-19T19:17:00+02:00

In short:

Buy whatever you fucking want.
Turn the fucking thing off when you're not using it.

Long:

People are spending a lot of time building an energy efficient home computer, that can act as an HTPC, NAS, or whatever. It must consume as little power as possible, because it it will be on 24/7.

Why the fuck do you want to leave a system on 24/7 at home?

Unless you're unemployed you will be at work most of the time. And during that time, this machine is doing absolutely nothing.

There may only be one system that stays on 24/7 and that is your router, either some embedded router thingy or something based on a low-power PC, but that's about it. Turn every thing else off.

A machine that does 200 Watt idle that is only turned on when necessary will be more energy efficient than your specially build 40 watt NAS or whatever it is that is running 24/7. Nothing can beat a system that is turned off.

If you need a system, just use wake-on-lan or WOL to turn the damn thing on. It will be ready in about 2 or 3 minutes and then you can do whatever you want.

Fine if you leave a system on that is downloading some stuff during the night or day, but after the download has finished, turn the damn thing off.

If you're honest with yourself and really think about it, there is no need to keep a system on for 24/7. I know you may come up with excuses, but remember that you still can use your router for that.

By the way, if you're searching for a really energy efficient computer, buy a Mac Mini. Although quite expensive if you ask me, the're doing 30 watt in idle and almost nothing when sleeping. And if a mac is asleep, it can be woken with WOL and is up in seconds. They make an excelent download server.

[EDIT]

There is also the option of S3 or S4 under Linux: suspend to ram or disk. However, your mileage may vary. If it works for you your system will be down and up in seconds. However, my experience is that it very much depends on your hardware if this will work. My Highpoint cards do not seem to like it, my array does not awake and the screen of the system stays blank.

If it works, it is a faster solution than just to turn the system off and on with WOL. My experience is that is seems not that robust.

Visual representation of hard drives and their temperature

2009-09-12T23:53:00+02:00

If you build a NAS with many drives, it may be of interest to you which drives get hot and where they are located in the chassis. My Norco 4020 case has twenty drives in RAID 6, plus two operating system drives in RAID 1. I wrote a script that shows me the temperature of each drive, positioned in such a way that it represents the actual physical location of the drive in the chassis:

 -- 30 -- | -- 27 --
 | 26 | 27 | 28 | 25 |
 | 30 | 32 | 30 | 29 |
 | 31 | 33 | 33 | 29 |
 | 33 | 32 | 35 | 30 |
 | 32 | 34 | 35 | 31 |

In this example, you can determine that the top drives seem to stay the coolest. The center and lower drives get hotter.

If you want to use this script, you will have to change it for your own specific setup. You can get it here.

This visual representation would also be nice to identify which drive has failed and where it is located in the chassis.

'Linux: obtain motherboard model / type and vendor'

2009-08-30T21:16:00+02:00

If you want to know what motherboard is installed in a system, use the tool dmidecode:

dmidecode | grep -e "Manufacturer|Product" | head -n 4 | tail -n 2

The result might be something like:

Manufacturer: ASUSTeK Computer INC.

Product Name: P5Q-EM DO

Manufacturer: ASUSTeK Computer INC.

Product Name: P6T DELUXE

Manufacturer: ASUSTeK Computer INC.

Product Name: M2A-VM

Manufacturer: Apple Computer, Inc.

Product Name: Mac-F4208EC8

Manufacturer: Compaq

Product Name: 0688h

'lm-sensors: hardware monitoring with the w83627ehf module'

2009-08-30T16:26:00+02:00

I have two systems that use the w83627ehf driver for hardware monitoring. However, if this driver is installed with a regular modprobe like:

modprobe w83627ehf

The result will be:

FATAL: Error inserting w83627ehf
(/lib/modules/2.6.28-1-amd64/kernel/drivers/hwmon/w83627ehf.ko): No such
device

I had this issue with the following mainboards:

Asus P5Q-EM DO (Core 2 duo) Asus: P6T deluxe (Core 7i)

The solution is simple, as I found after googling for some time:

modprobe w83627ehf force_id=0x8860

The result:

w83627ehf-isa-0290
Adapter: ISA adapter
VCore: +0.88 V (min = +0.00 V, max = +1.74 V)
in1: +11.67 V (min = +0.00 V, max = +5.49 V) ALARM
AVCC: +3.34 V (min = +0.40 V, max = +0.03 V) ALARM
3VCC: +3.31 V (min = +1.15 V, max = +0.58 V) ALARM
in4: +1.65 V (min = +0.54 V, max = +0.22 V) ALARM
in5: +2.04 V (min = +0.71 V, max = +0.11 V) ALARM
in6: +3.79 V (min = +0.10 V, max = +2.30 V) ALARM
VSB: +3.38 V (min = +0.45 V, max = +2.43 V) ALARM
VBAT: +3.30 V (min = +0.67 V, max = +0.77 V) ALARM
in9: +0.00 V (min = +1.10 V, max = +0.57 V) ALARM
Case Fan: 0 RPM (min = 104 RPM, div = 128) ALARM
CPU Fan: 0 RPM (min = 42187 RPM, div = 32) ALARM
Aux Fan: 0 RPM (min = 162 RPM, div = 128) ALARM
fan4: 0 RPM (min = 42187 RPM, div = 32) ALARM
fan5: 0 RPM (min = 42187 RPM, div = 32) ALARM
Sys Temp: +40.0°C (high = +4.0°C, hyst = +0.0°C) ALARM sensor = thermistor
CPU Temp: +42.0°C (high = +80.0°C, hyst = +75.0°C) sensor = diode
AUX Temp: +30.5°C (high = +80.0°C, hyst = +75.0°C) sensor = thermistor
cpu0_vid: +0.000 V

Using ncat to provide SSL-support to non-ssl capable software

2009-08-21T15:49:00+02:00

Sometimes, people are using software that does not support encrypted connections using SSL. To provide SSL-support to such a client, ncat can be used. Ncat is part of nmap, the famous port-scanner.

The main principle is that the non-ssl capable software does not connect to the SSL-based service, but to the local host. Ncat will be listening on the localhost and will setup an SSL-connection with the SSL-based service on behalf of the non-ssl capable software.

This simple command allows an application to browse to port 80, and perform regular HTTP-request, while in fact, they are encapsulated within a SSL- connection:

ncat -l 80 -c "ncat (ip-address) (port) --ssl"

The -l option specifies the local port on which the SSL-tunnel will be listening. The ip-address and port refer to the SSL-based service.

So if the client connects to 127.0.0.1 on port 80 it will actually connect through the SSL-tunnel to the external service.

Often stunnel is used for this job but this software craps out on debian Etch with some error like:

SSL routines:SSL3_GET_RECORD:bad decompression

But ncat is an excellent alternative.

Howto get the hard disk size under Linux?

2009-08-18T21:19:00+02:00

A: There is no single tool for this job, but it seems that Fdisk is just fine:

server:~# fdisk -l 2> /dev/null | grep Disk | grep -v identifier

Disk /dev/sda: 500.0 GB, 500028145664 bytes

Disk /dev/sdb: 500.0 GB, 500028145664 bytes

Disk /dev/sdc: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdd: 500.0 GB, 500028145664 bytes

Disk /dev/sde: 500.0 GB, 500028145664 bytes

Disk /dev/hda: 80.0 GB, 80026361856 bytes

Disk /dev/md5: 1500.0 GB, 1500084240384 bytes

The nice thing about using fdisk is that it automatically lists the size of all block devices.

Here is a list of my NAS, mentioned earlier.

Beast:~# fdisk -l 2> /dev/null | grep Disk | grep -v identifier

Disk /dev/sda: 60.0 GB, 60011642880 bytes

Disk /dev/sdb: 60.0 GB, 60011642880 bytes

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes

Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes

Disk /dev/sde: 1000.2 GB, 1000204886016 bytes

Disk /dev/sdf: 1000.2 GB, 1000204886016 bytes

Disk /dev/md0: 57.9 GB, 57996345344 bytes

Disk /dev/md1: 2015 MB, 2015100928 bytes

Disk /dev/sdg: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdh: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdi: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdj: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdk: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdl: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdm: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdn: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdo: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdp: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdq: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdr: 1000.1 GB, 1000123400192 bytes

Disk /dev/sds: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdt: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdu: 1000.1 GB, 1000123400192 bytes

Disk /dev/sdv: 1000.1 GB, 1000123400192 bytes

Disk /dev/md5: 18002.2 GB, 18002220023808 bytes

HighPoint RocketRAID and staggered spinup with Samsung F1

2009-07-21T20:13:00+02:00

I have experienced problems using a HighPoint RocketRaid 2320 and 2340 when they are using 'staggered spinup' in combination with Samsung Spinpoint F1, 1 (one) terrabyte disks.

The problem is that the F1 disks spinup very slowly and often seem to 'hang' while making ticking noises that will scare any computer user to death. When, after ages, you have the luck that no disks keeps hanging during startup, the first thing to do is to disable staggered spinup.

However, if staggered spinup is not used and all disks will spinup together, please note that you will need a strong PSU to handle the short peak load. For example, starting up 20 (twenty) disks will briefly generate a load of 550 Watt on my APC ups, according to its display.

I estimate that during startup, each disk is roughly consuming 20 Watt. Take this all into account when deciding which type of disks and controllers you will be using.

8 TB RAID 6 Linux software RAID using EXT4

2009-07-05T22:45:00+02:00

Mobo: Asus P5Q-EM DO ( 6x sata)

CPU: Core 2 Duo E7400

RAM: 4 GB

Controller; HighPoint RocketRAID 2314 (16x poorten)

PSU: CooMas Silent Pro M 600W ATX2 (definitely not overkill)

HD: Samsung 1 TB SAT2 HD103UJ (10x)

OS: Debian Lenny with custom kernel (backported)

FS: EXT4

RAID config: 10 disk RAID 6 based on Linux software RAID.

Performance: Read: 850 MB/s Write: 300 MB/s

Capacity: 7,5 GiB or 8 GB.

I am very pleased with the result. Points that should be improved are the temperature and noise. The PSU is a major overkill since this puppy draws less than 200 watt.

Bunny:~# ./show-hdd-temp.sh

Device Temperature

/dev/sda 32

/dev/sdb 34

/dev/sdc 33

/dev/sdd 36

/dev/sde 34

/dev/sdf 33

/dev/sdg 38

/dev/sdh 39

/dev/sdi 37

/dev/sdj 36

/dev/sdk 35

Personalities : [raid6] [raid5] [raid4]

md0 : active raid6 sdj[10] sdb[0] sdl[9] sdk[8] sdg[7] sdf[6] sdi[5] sdh[4] sde[3] sdd2 sdc1

7813463552 blocks super 0.91 level 6, 64k chunk, algorithm 2 [11/11] [UUUUUUUUUUU]

[======>..............] reshape = 30.1% (294904524/976682944) finish=585.3min speed=19413K/sec

Linux RAID 6 performance using software RAID

2009-06-28T19:32:00+02:00

So after toying around with RAID 0 just for fun, time to get serious. I created a RAID 6 of 10 x 1 TB disks. This gives me raw device read speeds of 850 MB/s and write speeds of 300 MB/s. I think this is exactly what should be expected, but boy it is damn fast. Especially the write speed surprises me.

Still, if I format the device using EXT4, write speed stays the same. However, read speed drop to about 350 MB/s. I don't understand why. Maybe some misalignment? I don't know.

The issue with the dropped read speed is due to an incorrect chunk size of the array. The array now consists of 20 drives, providing 18 TB of storage using XFS. Read speads exceed 1.1 GB/s (that is not a typo) and write speeds are about 350 MB/s.

1.0 GB/s using Linux software RAID

2009-06-26T23:58:00+02:00

I filled the Norco case with hardware. It is now up and running, based on Debian Linux (Lenny).

I immediately performed some initial tests with software RAID 0. The results are just astounding.

debian:~# dd if=/dev/md0 of=/dev/null bs=1M count=50000

50000+0 records in

50000+0 records out

52428800000 bytes (52 GB) copied, 50.9222 s, 1.0 GB/s

This is correct. 1 Gigabyte per second using 10 Samsung Spinpoint F1's connected to the motherboard (4) and to the Highpoint Rocketraid 2340 (6).

Bash Shell Function Library (BSFL) released.

2009-05-31T20:22:00+02:00

The Bash Shell Function Library (BSFL) is a small Bash script that acts as a library for bash scripts. It provides a couple of functions that makes the lives of most people using shell scripts a bit easier.

The purpose of this library is to provide pre-build functions for actions that often need to be performed, focussing on error-checking and logging. In this example, a test-script is written that demonstrates the functions the library provides.

The project can be found at this location.

'Tip of the day for every Linux or Unix user: brace expantion'

2009-05-02T22:24:00+02:00

Searching the web I discovered some really nice feature of the unix shell, which I didn't know about.

Try this:

touch foobar.conf

Now try this:

cp foobar.conf{,.bak}

It is equivalent to:

cp foobar.conf foobar.conf.bak

This is also the easiest way to create sequences. Do not use 'seq' since you cannot rely on it being installed.

bash-3.2$ echo {1..10}

1 2 3 4 5 6 7 8 9 10

Please visit this location for additional examples.

Compatibility Highpoint RocketRAID 2320 and Samsung Spinpoint F1

2009-04-26T22:38:00+02:00

There are some reports about incompatibility between RAID controllers and Samsung Spinpoint F1 drives. I have no troubles with my 0.5 and 1.0 TB drives from Samsung using mentioned controller. See below:

Controller 1: RocketRAID 232x SATA Controller

1/1/1 SAMSUNG HD103UJ 1000123MB, Normal

1/2/1 SAMSUNG HD103UJ 1000123MB, Normal

1/3/1 SAMSUNG HD103UJ 1000123MB, Normal

1/4/1 SAMSUNG HD103UJ 1000123MB, Normal

1/5/1 SAMSUNG HD501LJ 500028MB, Normal

1/6/1 SAMSUNG HD501LJ 500028MB, Normal

1/7/1 SAMSUNG HD501LJ 500028MB, Normal

1/8/1 SAMSUNG HD501LJ 500028MB, Normal

Distributed Parallel Processing Shell Script (PPSS) released

2009-03-12T22:05:00+01:00

I'd like to announce the release of the distributed version of the Parallel Processing Shell Script (PPSS). PPSS is a bash script that allows you to run commands in parallel. It is written to make use of current multi-core CPUs.

The new distributed version of PPSS allows you to run PPSS on multiple hosts, that simultaneously work on a single group of files or other items. A central server is used for file locking. This way, nodes know which files are 'locked' by other nodes and/or have already been processed.

PPSS is written to be very easy to use. You can be up and running on a single host within 5 minutes. Distributed usage requires the (one-time) setup of some SSH accounts on your nodes and server, but that's about it. Please take a look at distributed PPSS for yourself at:

http://code.google.com/p/ppss

I used distributed PPSS to encode 400 GB of WAV files to MP3 using 4 computer systems. A total of 14 cores where available to PPSS to encode those files using Lame. However, PPSS is not limited to encoding WAV files. To the contrary, it is written to execute whatever command you want to execute. Gzip a bunch of files in parallel? No problem. It is totally up to your imagination.

Did you know that RDP is as insecure as telnet?

2008-11-21T09:47:00+01:00

Microsoft developed the remote desktop protocol in order to allow remote GUI- based access to hosts. It is easy to pick on Microsoft and I do respect what they have accomplished, however, it may not come as a surprise that they did it all wrong with RDP in terms of security.

By default, out-of-the-box, a server or desktop supports RDP with RC4 encryption. Most recent servers and clients support 128-bits encryption. Sounds secure, right? Well, it isn't unfortunately. There are two major security weaknesses that affect the remote desktop protocol.

Keystroke vulnerability

The original , RDP version 4.0, is based on the T.128 protocol. Input events, such as keystrokes, are sent to the server with a unique timestamp as specified in the T.128 standard. This means that each message containing an input event is unique and so will be the checksum generated from this input event data. In other words: if two RDP messages contain the same data, they will still be unique because of the timestamp. Microsoft did it right in RDP version 4.0, but there was one drawback: the timestamp generated overhead, making RDP messages relatively large.

So in RDP version 5.0, Microsoft decided to drop the timestamp. Thus, if two RDP messages contain the same data, their checksum will be identical. So if the same key is pressed twice, the two packets that are generated will contain different encrypted data segments but the checksums will be identical. Although it is not clear which key is pressed, if a larger data set is analyzed, patterns can be discovered and an accurate guess can be made to wich keys are pressed.

In a nuttshell an attacker that has access to the data stream of an RDP session will be able to decode keystrokes and thus be able to:

obtain your username and password, allowing access to the system!
read what you type.

Please visit the following location as it provides a more detailed explanation of this vulnerability:

http://www.xatrix.org/article.php?s=1943

Man-in-the-middle-attack

At first, back in 2003, RDP-clients did not verify the identity of the RDP server. So an attacker can easily spoof an identity and perform a man-in-the- middle-attack. Microsoft attempted to fix this vulnerability but did not succeed. Although more recent RDP-clients do verify the identify of the server, it is still possible to spoof this identity. Microsoft uses a fixed hard-coded key to identify the server with. This key resided in a system DLL that is available to the general public. An attacker can easily obtain this key, create a fake identity and still prove to be legitimate since the identity is signed with a trusted key.

More detailed information:

http://www.oxid.it/downloads/rdp-gbu.pdf

Conclusion

Given these vulnerabilities, RDP is, in its default configuration, virtually as insecure as plain old telnet. Think about that the next time you logon with administrator credentials using RDP.

Tool that implements the attacks

The well-known hack-tool Cain & Abel has a build-in RDP man-in-the-midlle implementation. It automatically decrypts (parts of) RDP data packets and reveals the key strokes that are send to a server. So if you don't believe this story and just want to see for yourself how easy such an attack really is, download this tool and test it for yourself.

To defend against the attacks

To protect against this attack, it is strongly advised to implement RDP based on TLS/SSL encryption. Ofcourse, you'll have to install a server-side SSL- certificate and it may be necessary to obtain an official (as in not self-) signed certificate.

http://technet2.microsoft.com/windowsserver/en/library/a92d8eb9-f53d-4e86-ac9b-29fd6146977b1033.mspx?mfr=true

So please understand the risks of using RDP, especially over the Internet. If TLS is for some reason not an option, use an encrypted VPN-connection to tunnel the RDP-session in, as an added layer of protection. Never use RDP in its default configuration over the interne without additional protection.

EDIT: IN RDP version 6.0 the man-in-the-middle-attack is no longer possible!

EDIT2: Configure your terminal server to use FIPS compliant encryption. It will use Triple DES encryption even if TLS is not enabled.

After you enable this setting on a Windows Server 2003-based computer, the following is true:

The RDP channel is encrypted by using the 3DES algorithm in Cipher Block Chaining (CBC) mode with a 168-bit key length.

The SHA-1 algorithm is used to create message digests.

Clients must use the RDP 5.2 client program or a later version to connect.

So you don't need to use TLS if you want to use a more secure algorithm than RC4.

4e86-ac9b-29fd6146977b1033.mspx?mfr=true

The kind of sound you never want to hear from your computer

2008-11-14T21:16:00+01:00

The sounds of hard drives that have perished

These sounds are only frightening or scary if you imagine your precious data only exists on that failing drive. If you make consistent and frequent backups and/or you run a fault-tolerant RAID flavour, it is but a nuisance.