Bringing chat to the browser with JWChat

If you just want to use JWChat to connect to Jabber without downloading any software or performing any server setup, you can visit jwchat.org. I tested jwchat.org with a connection to jabber.se from a browser and using Pidgin to verify communication, and it worked fine, but connecting through that site does have drawbacks. Installing JWChat on your own server lets you control the security and availability of the server, integrate JWChat with your site, and establish your own Jabber server on your LAN.

Given that AJAX allows a Web page to issue requests from a Web browser in the background, you might at first think that JWChat would just use AJAX to directly connect to your Jabber server of choice. But the JavaScript XMLHttpRequest function that forms the core of an AJAX application is subject to a security restriction such that it can contact only the same site that served up the HTML page from which it was called. To get around this, on the server that you load JWChat from you have to either (a) use Apache RewriteRules, (b) install a Jabber server that handles HTTP polling or binding, or (c) install an application proxy on the machine that serves up JWChat, such as PunJab. So while the actual Jabber client itself, JWChat, is running inside your Web browser, you need something on the server from which you loaded JWChat that is either a Jabber server or that acts as an intermediary between JWChat and other Jabber servers.

Scientific study about Debian Project governance and social organization

Andr� Felipe Machado SPAMFILTER@techforce.com.br>

Ter�a-Feira, 15 de Abril de 2008

Kernel space: memory allocation failures

By Jonathan Corbet, LinuxWorld.com, 04/15/08

If your production Linux system is logging memory allocation failures, it might still be able to keep running. But developers want to keep an eye on which code can survive a shortage of memory.

In general, the kernel's memory allocator does not like to fail. So, when kernel code requests memory, the memory management code will work hard to satisfy the request. If this work involves pushing other pages out to swap or removing data from the page cache, so be it. A big exception happens, though, when an atomic allocation (using the GFP_ATOMIC flag) is requested. Code requesting atomic allocations is generally not in a position where it can wait around for a lot of memory housecleaning work; in particular, such code cannot sleep. So if the memory manager is unable to satisfy an atomic allocation with the memory it has in hand, it has no choice except to fail the request.

Such failures are quite rare, especially when single pages are requested. The kernel works to keep some spare pages around at all times, so the memory stress must be severe before a single-page allocation will fail. Multi-page allocations are harder, though; the kernel's memory management code tends to fragment pages, making groups of physically-contiguous pages hard to find. In particular, if the system is under pressure to the point that there is not much free memory available at all, the chances of successfully allocating two (or more) contiguous pages drops considerably.

Multi-page allocations are not often used in the kernel; they are avoided whenever possible. There are situations where they are necessary, though. One example is network drivers which (1) support the transmission and reception of packets too large to fit into a single page, and which (2) drive hardware which cannot perform scatter/gather I/O on a single packet. In this situation, the DMA buffers used for packets must be larger than one page, and they must be physically contiguous. This is a situation which will become less pressing over time; scatter/gather capability in the hardware is increasingly common, and drivers are being rewritten to make use of this capability. With sufficiently smart hardware, the need for multi-page allocations goes down considerably.

But all of that skirts around the main point, which is that kernel code is supposed to handle allocation failures properly. There is never any guarantee that memory will be available, so kernel code must be written defensively. Allocation failures must be handled without losing any more capability than is strictly necessary. If one assumes that kernel code is written correctly, there should be no need to issue warnings on allocation failures. Things should just continue to work, perhaps without users noticing at all.

And, in fact, things often do just work. But the discussion resulting from Dave's suggestion makes it clear that few developers are confident that all kernel code does the right thing in the face of memory allocation problems. In cases where an allocation failure is not dealt with correctly, the system may go down in random places, leaving few clues as to what really happened. In that kind of situation, the allocation failure warning may be the only useful information which survives the crash. For this reason, some people want to see the warnings left in place.

As it happens, the memory allocator supports a special bit (__GFP_NOWARN) which causes the warning not to be emitted if a specific allocation fails. So it has been suggested that the allocations made from code which is known to handle failures properly have __GFP_NOWARN set. That would kill the warnings in code known to do the right thing while leaving it for all other callers, presumably limiting the warnings to places where there might truly be a problem. Jeff Garzik strongly opposed this idea, though, saying that it clutters up the code and "punishes good behavior."

The other reason given for keeping the warnings in place is to make it clear when a system is running under persistent memory pressure. Such systems will not be performing optimally; often there are changes which can be made to relieve the pressure and help the system to run more smoothly. So it has been suggested that the warning could be reduced in frequency and made less scary. Nick Piggin suggests:



So I think that the messages should stay, and they should print out some header to say that it is only a warning and if not happening too often then it is not a problem, and if it is continually happening then please try X or Y or post a message to lkml...

An alternative idea would be to keep some sort of counter somewhere which could be queried by curious system administrators.

Of course, the real solution is to ensure that all kernel code is robust in the face of allocation failures. This can be hard to do, since the error recovery paths in any code are not often exercised or tested. Fortunately, the fault injection framework can help in this situation. Kernel developers can use this framework to simulate allocation failures in specific regions of code, then watch to see what happens. The author's impression, though, is that relatively few developers are using this tool. So confidence in the kernel's handling of allocation failures may remain low, and the desire to keep the warning around may remain high.

Original link: http://www.linuxworld.com/news/2...

Metamodeling with EMF: Generating concrete, reusable Java snippets

Level: Intermediate

Ken McNeill (ken.mcneill@fr.ibm.com), Consulting Solution Architect, IBM

08 Apr 2008

Learn how to extend the Eclipse Modeling Framework (EMF) Ecore metamodel by adding elements and attributes to model reusable Java™ snippets. We will also see, step by step, how to use dynamic templates with JET to generate the implementation code for the extended model elements.

EMF is an integral part of the Eclipse platform, as well as a cornerstone of related technologies and frameworks, such as the Eclipse Visual Editor, SDO, XSD, and UML — many of which are integrated into IBM� platforms like Rational� Application Developer and WebSphere� Business Modeler. Today, EMF has grown to encompass Java technology features, such as enumerated types, annotations, and generics. If you are new to EMF, see Resources for articles that will help you get started.

In most documents and tutorials, EMF is used to model data and interfaces (e.g. Library and Books in the EMF release documentation), and not behavior. Of course, there are some default method implementations generated for data objects, but these concern relationships between model elements. Moreover, there are very few documented examples of EMF being used as a "meta-metamodel" — with the exception of the Eclipse Foundation article "Modeling Rule-Based Systems with EMF" (see Resources) — but not a single example showing how to extend the Ecore metamodel.

Wizard Boot Camp, Part One

Over the past five years, this column has covered a lot of not-so-obvious features of Linux — mostly techniques that you’d use from the shell or another non-graphical environment. The start of your columnist’s sixth year seemed like a good time to try something a little different: a series of columns on features of Linux and shells that many books about Linux basically ignore, but that wizards should know. Studying these obscure features isn’t just for trivia buffs. Understanding how each feature works can help you grok “the big picture” of some pieces that make your favorite operating system the power platform that it is.

This month, let’s start with a quick overview of Linux and shell concepts that you should know, then dig into some shell features that might be new to you.

Linux Overview in Ten Paragraphs

Let’s review some of the fundamentals to start from the same place. (Please note: this isn’t an exact, complete description. Some details are simplified or omitted.)

A Linux system manages a set of processes, letting them share system resources (the processors, disks, networks, and so on). Basically, a process is a running program, a program waiting to run, or a suspended program. A process has a name (often the name of the program file that it’s executing) and a list of arguments (for instance, some filenames). Each process has a unique process ID, or PID, that’s assigned when the process starts. The ps utility lists processes.

A process runs under the user ID, or UID, of one of the accounts on the system. In most cases, a process runs with the UID of the account it was started under. So, if user joe starts a text editor, that process inherits the abilities that joe has: for instance, it can read and write his files. The superuser, root, has UID 0. A process running under UID 0 is privileged: it can change its UID (actually, its effective UID) to any other UID on the system. The su and sudo programs, which run under UID 0, allow any user to run processes as another user.

A parent process can start other processes called child processes. A child process inherits many of its parent’s attributes — as you’ll see next. A child process can also start its own child processes. The process named init, with PID 1, is the parent (or “grandparent”, or great-grandparent, and so on) of all processes. After a parent process starts a child process, it typically waits for the child to finish. However, if the parent doesn’t wait, it can “disown” the child, and that “orphaned” process becomes a child of init.

Boot Camp for Wizards, Part Two

This series started with an overview of Linux fundamentals and a look at some important concepts behind shells. Let’s continue looking at obscure features of Linux that books generally don’t cover but that wizards should know. Read on to find out more about the standard I/O system. Though the ideas behind standard I/O are common knowledge, let’s start with an introduction to get on the same page (so to speak).

What’s behind standard I/O?

Back in the days when Unix was new, the idea of standard I/O wasn’t commonplace. It wasn’t necessarily simple to write data to different devices - a program might need different code to read or write files, tapes, terminals and printers. Users were accustomed to seeing progress messages and summaries.

A well-behaved Unix/Linux utility outputs only valid data. If there’s no trouble and no data, there’s no output. For instance, if grep doesn’t find matching text, it doesn’t output “No match found.” It simply terminates.

Unix popularized the idea that data can be a stream of bytes, that a program reads and writes without knowing where the data is coming from or where it’s going. The operating system and its device drivers convert a stream of bytes to and from the correct formats for all of the system’s devices (disks, printers, tape drives, and so on), letting the programs handle the data. The Unix operating system and the shell also arrange the “plumbing” that routes data between processes and devices - for instance, by doing the I/O buffering and process stops and starts that make a pipe work. Linux has the same concepts, of course.

As much as possible, a general-purpose Linux program should be able to read data from its standard input (stdin), write results to its standard output (stdout), and write warnings or errors to the standard error (stderr). The stdout of a general-purpose program should be only data - data that’s ready for writing to a file, sending down a network, or reading by another program. A general-purpose program shouldn’t write progress messages to stdout (”reading somefile… ” or “23% done”) because, if it did, other programs would need to distinguish between that “out-of-band” text and valid data. Errors and warnings also shouldn’t go to stdout because stdout may be redirected to a file, a printer, or another device. Instead, warnings and errors should go to stderr, which almost always goes to the user interface - typically, a terminal - where the messages will appear.

Boot Camp for Wizards, Part Three: Using Standard I/O from shells

Last month’s column, the second in this series of obscure Linux features that wizards should know, introduced Standard I/O. This month we’ll see how to take advantage of Standard I/O from a shell– including an example using named pipes (FIFOs).

Duplicating a file descriptor

The Bourne shells’ operator m>&n copies a file descriptor. The first digit, m, is the file descriptor you want to change. The second digit, n, is the file descriptor to copy.

For instance, 2>&1 copies f.d. 2 from f.d. 1. This makes f.d. 2 point to the same open file as f.d. 1 does. Because f.d. 2 is the standard error and f.d. 1 is standard output, after using that operator, stderr will point to the same place as stdout.

The operator 2>&1 isn’t any use in the default situation where both stderr and stdout point to the terminal. One place it is handy is when you’re piping the output of one command to another– for instance, when you’re viewing the output of a program with a pager program like less (1). Without the 2>&1 operator, like this:

$ grep xyz * | less

Only the stdout of grep is piped to less. grep‘ s stderr goes to the terminal– along with the stdout of less. So, if grep outputs an error, it’s likely to be mixed into the output text from less; you may not notice the error at all. Using 2>&1 fixes that:

$ grep xyz * 2>&1 | less

With that operator, grep‘ s stderr goes the same place as grep‘ s stdout: down the pipe, to be read by (and paged by) less. bash sets up the pipe first– connecting grep‘ s stdout to the stdin of less– then it redirects grep‘ s stderr. The 2>&1 must be written to the left side of the pipe operator; it applies to the command on the left-hand side.

The order of redirections is important: the shell reads a command line and processes redirections from left to right.

Wizard Boot Camp, Part Four: Tracking Processes

You probably already know you can examine process status with ps, but most users never embrace the full power of ps. I’ve been talking about Linux concepts that wizards should know, but books often gloss over, and ps is at the top of the list. Let’s dig into processes using ps and some of its more useful options.

Some Basics

The ps utility lists some or all of your system’s current processes. By default– when you just run ps — you’ll only see processes owned by you (running under your UID). Whether ps shows some or all of the processes, for all users or only for you, depends on its configuration and on your system. For instance, ps on a shared Web host may prevent non-root users from seeing other users’ processes– even when using the “all” (-A) option.

The output of ps can also vary from system to system and kernel to kernel. Running ps on a multiprocessor system, for instance, can show which processor a job is running on. (To find out how many processors your system has and their characteristics, try running less /proc/cpuinfo. We’ll cover more of the /proc filesystem in a later column.)

To find out more about your ps, read its man page and compare its output on various systems. The GNU version understands three different option syntaxes and has the usual long list of GNU features. Because the BSD (Berkeley) options (which don’t start with a dash) give different output than the more-familiar Unix-style options (which do start with a dash), let’s focus on them. Simple ps output can look like this:

$ ps x
  PID TTY    STAT  TIME COMMAND
10445 ?      S     0:00 sshd: jpeek@pts/0
10450 pts/0  Ss    0:00 -bash
10499 ?      S     0:00 sshd: jpeek@pts/1
10501 pts/1  Ss    0:00 -bash
10508 pts/1  R     0:00 ps x

Wizard Boot Camp, Part Five: Control Processes Using Signals

A true wizard doesn’t just run processes, a true wizard knows how to communicate with those processes through signals to stop, restart, and even kill processes. This may sound like arcane and obscure knowledge, but if you follow along, you’ll be managing your processes like an adept in no time.

The first articles in this series of things wizards should know (but books often don’t cover) explained some Linux and shell fundamentals, discussed standard input/output and other open files, covered redirection of standard I/O and higher-numbered file descriptors, touched on FIFOs and a few other tips. Last month’s column covered ps and some of what it can disclose about processes.

This month we’ll dig into process control: signals sent to processes and how a shell handles processes, including some details on job control.

Introducing signals

Since the early days of Unix, users have terminated processes by sending a signal with kill. Some signals don’t actually” kill” a process: they notify the process (to, say, re-read its configuration file), suspend the process, or restart it. The first argument to kill is the signal you want to send; the other arguments are either the job numbers (see below) or PIDs of the process (es) that should get the signal. You can also send a signal to the foreground shell process by typing with key combinations like CTRL-C and CTRL-Z on your keyboard, as we’ll see soon.

Note that, in general, you can only send signals to your own processes. Only the root user is allowed to send a signal to any process running on the system.

Each signal has both a numeric value and a name. Some signals have the same value and name on almost every system. For instance, signal 15, named SIGTERM, is the default signal sent by the kill utility. Signal 2, SIGINT, is the” interrupt from keyboard” signal that’s typically sent by pressing CTRL-C. (Use stty to change the default signals sent by various key combinations during your login session.)

The list of signals varies in different versions of Unix and Linux, and some signals are system-specific. See the man page for signal and the header file (signal.h), for specifics of signals on your system.###

Linux examined: Ubuntu Hardy Heron

April 15, 2008 (Computerworld) If there is a single complaint that is laid at the feet of Linux time and time again, it's that the operating system is too complicated and arcane for casual computer users to tolerate. You can't ask newbies to install device drivers or recompile the kernel, naysayers argue.

Of course, many of those criticisms date back to the bad old days, but Ubuntu, the user-friendly distribution sponsored by Mark Shuttleworth's Canonical Ltd., has made a mission out of dispelling such complaints entirely.

You can now download a beta of Ubuntu's 8.04 release, more commonly and affectionately known as Hardy Heron (the follow-up to Gutsy Gibbon and Feisty Fawn). Final release is set for April 24.

At a Glance

Pros: Excellent desktop distribution that will seamlessly install on just about any hardware.

Cons: Not a first choice for servers.

Who should use this: Anyone who wants Linux on a desktop.
Hardy is what is known as an LTS (long-term support) release, meaning that patches and paid support will be available for at least three years after the release. Canonical has been dropping new releases about twice a year, ensuring that the kernel and software packages stay fresh. There are a lot of neat new features in Hardy, but let's start by talking about what makes Ubuntu such a great distribution to begin with.

Ubuntu is based on Debian, which enjoys wide developer support. Having a vibrant distribution such as Debian as Ubuntu's underpinning has resulted in a very stable and feature-rich distribution. Raw Debian has a reputation as being a bit geek-centric, and although it makes a good effort to be easy to use, it still can be a challenge to install for nontechnical users. Ubuntu has put padding on a lot of Debian's sharp corners, without removing any of the power of the underlying distribution.

One of the killer features of Ubuntu is that the installation media is also a "live CD." This means that you can boot it off the CD and try it out first before installing it. (You can also set Ubuntu up to boot off a USB drive.) In other words, you can make sure that all your hardware will work correctly and that you're happy with the look and feel of the operating system before committing yourself to anything permanent. You can also carry it around and use it to boot up a friend's computer under Ubuntu.

And when you do install it, you'll be asked a minimum of questions, and none of them are in the least challenging to anyone who has ever installed Windows. The install is even smart enough to help you resize an existing Windows partition (even Vista!) to set up a dual-boot system and set the boot menu to handle it.
Ubuntu
Ubuntu 8.04, known as Hardy Heron.
Click to view larger image.