IT SOLUTIONS

Hack 13. Understand Visual Processing

2008-06-02T05:42:00.000-07:00

The visual system is a complex network of modules and pathways, all specializing in different tasks to contribute to our eventual impression of the world.

When we talk about "visual processing," the natural mode of thinking is of a fairly self-contained process. In this model, the eye would be like a video camera, capturing a sequence of photographs of whatever the head happens to be looking at at the time and sending these to the brain to be processed. After "processing" (whatever that might be), the brain would add the photographs to the rest of the intelligence it has gathered about the world around it and decide where to turn the head next. And so the routine would begin again. If the brain were a computer, this neat encapsulation would be how the visual subsystem would probably work.

With that (admittedly, straw man) example in mind, we'll take a tour of vision that shows just how nonsequential it all really is.

And one need go no further than the very idea of the eyes as passive receptors of photograph-like images to find the first fault in the straw man. Vision starts with the entire body: we walk around, and move our eyes and head, to capture depth information [Hack #22] like parallax and more. Some of these decisions about how to move are made early in visual processing, often before any object recognition or conscious understanding has come into play.

This pattern of vision as an interactive process, including many feedback loops before processing has reached conscious perception, is a common one. It's true there's a progression from raw to processed visual signal, but it's a mixed-up, messy kind of progression. Processing takes time, and there's a definite incentive for the brain to make use of information as soon as it's been extracted; there's no time to wait for processing to "complete" before using the extracted information. All it takes is a rapidly growing dark patch in our visual field to make us flinch involuntarily [Hack #32], as if something were looming over us. That's an example of an effect that occurs early in visual processing.

But let's look not at the mechanisms of the early visual system, but how it's used. What are the endpoints of all this processing? By the time perception reaches consciousness, another world has been layered on top of it. Instead of seeing colors, shapes, and changes over time (all that's really available to the eyes), we see whole objects. We see depth, and we have a sense of when things are moving. Some objects seem to stand out as we pay attention to them, and others recede into the background. Consciously, we see both the world and assembled result of the processing the brain has performed, in order to work around constraints (such as the eyes' blind spot [Hack #16] ), and to give us a head start in reacting with best-guess assumptions. The hacks in this chapter run the whole production line of visual processing, using visual illusions and anomalies to point out some detail of how vision works.

But before diving straight into all that, it's useful to have an overview of what's actually meant by the visual system. We'll start at the eye, see how signals from there go almost directly to the primary visual cortex on the back of the brain, and from there are distributed in two major streams. After that, visual information distributes and merges with the general functions of the cortex itself.

2.2.1. Start at the Retina

In a sense, light landing on the retinathe sensory surface at the back of the eyeis already inside the brain. The whole central nervous system (the brain and spinal column [Hack #7]) is contained within a number of membranes, the outermost of which is called the dura mater. The white of your eye, the surface that protects the eye itself, is a continuation of this membrane, meaning the eye is inside the same sac. It's as if two parts of your brain had decided to bulge out of your head and become your eyes, but without becoming separate organs.

The retina is a surface of cells at the back of your eye, containing a layer of photoreceptors, cells that detect light and convert it to electrical signals. For most of the eye, signals are aggregateda hundred photoreceptors will pass their signal onto a single cell further along in the chain. In the center of the eye, a place called the fovea, there is no such signal compression. (The population density of photoreceptors changes considerably across the retina [Hack #14] .) The resolution at the fovea is as high as it can be, with cells packed in, and the uncompressed signal dispatched, along with all the other information from other cells, down the optic nerve. The optic nerve is a bundle of projections from the neurons that sit behind the photoreceptors in the retina, carrying electrical information toward the brain, the path of information out of the eye. The size of the optic nerve is such that it creates a hole in our field of vision, as photoreceptors can't sit over the spot where it quits the eyeball (that's what's referred to as the blind spot [Hack #16] ).

2.2.2. Behind the Eyes

Just behind the eyes, in the middle, the optic nerves from each eye meet, split, and recombine in a new fashion, at the optic chiasm. Both the right halves of the two retinas are dispatched to the left of the brain and vice versa (from here on, the two hemispheres of the brain are mirror images of each other). It seems a little odd to divide processing directly down the center of the visual field, rather than by eye, but this allows a single side of the brain to compare the same scene as observed by both eyes, which it needs to get access to depth information.

The route plan now is a dash from the optic chiasm right to the back of the brain, to reach the visual cortex, which is where the real work starts happening. Along the way, there's a single pit stop at a small region buried deep within the brain called the lateral geniculate nucleus, or LGN (there's one of these in each hemisphere, of course).

Already, this is where it gets a little messy. Not every signal that passes through the optic chiasm goes to the visual cortex. Some go to the superior colliculus, which is like an emergency visual system. Sitting in the midbrain, it helps with decisions on head and eye orienting. The midbrain is an evolutionary, ancient part of the brain, involved with more basic responses than the cortex and forebrain, which are both better developed in humans. (See [Hack #7] for a quick tour.) So it looks as if this region is all low-level functioning. But also, confusingly, the superior colliculus influences high-level functions, as when it suddenly pushes urgent visual signals into conscious awareness [Hack #37] .

Actually, the LGN isn't a simple relay station. It deals almost entirely with optical information, all 1.5 million cells of it. But it also takes input from areas of the brain that deal with what you're paying attention to, as well as from the cortex in general, and mixes that in too. Before visual features have as been extracted from the raw visual information, sophisticated input from elsewhere is being addedwe're not really sure of what's happening here.

There's another division of the visual signal here, too. The LGN has processing pathways for two separate signals: coarse, low-resolution data (lacking in color) goes into the magnocellular pathway. High-resolution information goes along the parvocellular pathway. Although there are many subsequent crossovers, this division remains throughout the visual system.

2.2.3. Enter the Visual Cortex

From the LGN, the signals are sent directly to the visual cortex. At the lower back of the cerebrum (so about a third of the way up your brain, on the back of your head, and toward the middle) is an area of the cortex called either the striate or primary visual cortex. It's called "striate" simply because it contains a dark stripe when closely examined.

Why the stripes? The primary visual cortex is literally six layers of cells, with a thicker and subdivided layer four where the two different pathways from the LGN land. These projections from LGN create the dark band that gives the striate cortex its name. As visual information moves through this region, cells in all six layers play a role in extracting different features. It's way more complex than the LGNthe striate contains about 200 million cells.

The first batch of processing takes place in a module called V1. V1 holds a map of the retina as source material, which looks more or less like the area of the eye it's dealing with, only distorted. The part of the map that represents the foveathe high-resolution center of the eyeis all out of proportion because of the number of cells dedicated to it. It's as large as the rest of the map put together.

Physically standing on top of this map are what are called hypercolumns. A hypercolumn is a stack of cells performing processing that sits on top of an individual location and extracts basic information. So some neurons will become active when they see a particular color, others when they see a line segment at a particular angle, and other more complex ones when they see lines at certain angles moving in particular directions. This first map and its associated hypercolumns constitute the area V1 (V for "vision"); it performs really simple feature extraction.

The subsequent visual processing areas named V2 and V3 (again, V for "vision," the number just denotes order), also in the visual cortex, are similar. Information gets bumped from V1 to V2 by dumping it into V2's own map, which acts as the center for its batch of processing. V3 follows the same pattern: at the end of each stage, the map is recombined and passed on.

2.2.4. "What" and "Where" Processing Streams

So far visual processing has been mostly linear. There are feedback (the LGN gets information from elsewhere on the cortex, for example) and crossovers, but mostly the coarse and fine visual pathways have been processed separately and there's been a reasonably steady progression from the eye to the primary visual cortex.

From V3, visual information is sent to dozens of areas all over the cortex. These modules send information to one another and draw from and feed other areas. It stops being a production line and turns into a big construction site, with many areas extracting and associating different features, all simultaneously.

There's still a broad distinction between the two pathways though. The coarse visual information, the magnocellular pathway, flows up to the top of the head. It's called the dorsal stream, or, more memorably, the "where" stream. From here on, there are modules to spot motion and to look for broad features.

The fine detail of vision from the parvocellular pathway comes out of the primary visual cortex and flows down the ventral streamthe "what" stream. The destination for this stream is the inferior temporal lobe, the underside of the cerebrum, above and behind the eyes.

As the name suggests, the "what" stream is all about object recognition. On the way to the temporal lobe, there's a stop-off for a little further processing at a unit called the lateral occipital complex (LOC). What happens here is key to what'll happen at the final destination points of the "what" stream. The LOC looks for similarity in color and orientation and groups parts of the visual map together into objects, separating them from the background.

Later on, these objects will be recognized as faces or whatever else. It represents a common method: the visual information is processed to look for features. When found, information about those features is added to the pool of data, and the whole lot is sent on.

2.2.5. Processing with Built-in Assumptions

The wiring diagram for all the subsequent motion detection and object recognition modules is enormously complex. After basic feature extraction, there's still number judgment, following moving objects, and spotting biological motion [Hack #77] to be done. At a certain point, the defining characteristic of the cortex as a whole must come into play, and visual information is processed enough to be associated with memory, language, and reading emotions. This is where it blends in to the higher-order functions of the whole brain.

In the hacks that follow, we'll explore the effects of early and late visual processing. A common thread through these effects will be the assumptions the visual system has made about the visual world to expedite its computationand by looking at the quirks of vision, we can draw some of these out. Assumptions like the visual world remaining relatively stable from second to second (so we don't notice if it doesn't [Hack #40] ) and supposing that dark areas are shadows, which is the quirk that makeup takes advantage of [Hack #20] .

In a sense, the fact that we can observe these assumptions suggests that the visual system assumes as much about the external environment as about its own modules. The visual system's expectation that the motion module will report motion correctly (and therefore our confusion when the module doesn't identify motion correctly [Hack #25] ) is much the same as the visual system's expectation that a shadow is reporting 3D shape correctly. While we could think of the visual system as entirely in the brain, really we should include the eyes, the head, the body, and the environment as components in this big, messy, densely connected human visual processing system, all of which report their conclusions into the mix.

And somehow, in all of this, the visual perception we know and love somehow springs into existence. There doesn't seem to be a single place where all this visual processing is reassembled, no internal television screen that we watch (and even if there were, who would watch it?). It's distributed over the whole visual system, and over the environment too. Not just a picture at the retina, after all.

Hack 12. Build Your Own Sensory Homunculus

2008-06-02T05:41:00.000-07:00

All abilities are skills; practice something and your brain will devote more resources to it.

The sensory homunculus looks like a person, but swollen and out of all proportion. It has hands as big as its head; huge eyes, lips, ears, and nose; and skinny arms and legs. What kind of person is it? It's you, the person in your head. Have a look at the sensory homunculus first, then make your own.

1.13.1. In Action

You can play around with Jaakko Hakulinen's homunculus applet (http://www.cs.uta.fi/~jh/homunculus.html; Java) to see where different bits of the body are represented in the sensory and motor cortex.

This is the person inside your head. Each part of the body has been scaled according to how much of your sensory cortex is devoted to it. The area of cortex responsible for processing touch sensations is the somatosensory cortex. It lives in the parietal lobe, further toward the back of the head than the motor cortex, running alongside it from the top of the head down each side of the brain. Areas for processing neighboring body parts are generally next to each other in the cortex, although this isn't always possible because of the constraints of mapping the 3D surface of your skin to a 2D map. The area representing your feet is next to the area representing your genitals, for example (the genital representation is at the very top of the somatosensory cortex, inside the groove between the two hemispheres).

The applet lets you compare the motor and sensory maps. The motor map is how body parts are represented for movement, rather than sensation. Although there are some differences, they're pretty similar. Using the applet, when you click on a part of the little man, the corresponding part of the brain above lights up. The half of the man on the left is scaled according to the representation of the body in the primary motor cortex, and the half on the right is scaled to represent the somatosensory cortex. If you click on a brain section or body part, you can toggle shading and the display of the percentage of sensory or motor representation commanded by that body part. The picture of the man is scaled, too, according to how much cortex each part corresponds to. That's why the hands are so much larger than the torso.

Having seen this figure, you can see the relative amount of your own somatosensory cortex devoted to each body part by measuring your touch resolution. To do this, you'll need a willing friend to help you perform the two-point discrimination test.

Ask your friend to get two pointy objectstwo pencils will doand touch one of your palms with both of the points, a couple of inches apart. Look away so you can't see him doing it. You'll be able to tell there are two points there. Now get your friend to touch with only one pencilyou'll be able to tell you're being touched with just one. The trick now is for him to continue touching your palm with the pencils, sometimes with both and sometimes with just one, moving the tips ever closer together each time. At a certain point, you won't be able to tell how many pencils he's using. In the center of your palm, you should be able to discriminate between two points a millimeter or so apart. At the base of your thumb, you've a few millimeters of resolution.

Now try the same on your backyour two-point discrimination will be about 4 or 5 centimeters.

To draw a homunculus from these measurements, divide the actual width of your body part by the two-point discrimination to get the size of each part of the figure.

My back's about 35 centimeters across, so my homunculus should have a back that's 9 units wide (35 divided by 4 centimeters, approximately). Then the palms should be 45 units across (my palm is 9 centimeters across; divide that by 2 millimeters to get 45 units). Calculating in units like this will give you the correct scalesthe hand in my drawing will be five times as wide as the back.

That's only two parts of your body. To make a homunculus like the one in Hakulinen's applet (or, better, the London Natural History Museum's sensory homunculus model: http://owen.nhm.ac.uk/piclib/www/image.php?img=87494&cat=6), you'll also need measurements all over your face, your limbs, your feet, fingers, belly, and the rest. You'll need to find a fairly close friend for this experiment, I'd imagine.

1.13.2. How It Works

The way the brain deals with different tactile sensations is the way it deals with many different kinds of input. Within the region of the brain that deals with that kind of input is a surface over which different values of that input are processeddifferent values correspond to different actual locations in physical space. In the case of sensations, the body parts are represented in different parts of the somatosensory cortex: the brain has a somatotopic (body-oriented) map. In hearing, different tones activate different parts of the auditory cortex: it has a tonotopic map. The same thing happens in the visual system, with much of the visual cortex being organized in terms of feature maps comprised of neurons responsible for representing those features, ordered by where the features are in visual space.

Maps mean that qualities of stimuli can be represented continuously. This becomes important when you consider that the evidence for each qualityin other words, the rate at which the neurons in that part of the map are firingis noisy, and it isn't the absolute value of neural firing that is used to calculate which is the correct value but the relative value. (See [Hack #25] on the motion aftereffect for an example of this in action.)

The more cells the brain dedicates to building the map representing a sense or motor skill, the more sensitive we are in discriminating differences in that type of input or in controlling output. With practice, changes in our representational maps can become permanent.

Brain scanning of musicians has shown that they have larger cortical representations of the body parts they use to play their instruments in their sensory areasmore neurons devoted to finger movements among guitarists, more neurons devoted to lips among trombonists. Musicians' auditory maps of "tone-space" are larger, with neurons more finely tuned to detecting differences in sounds,¹ and orchestra conductors are better at detecting where a sound among a stream of other sounds is coming from.

It's not surprising that musicians are good at these things, but the neuroimaging evidence shows that practice alters the very maps our brains use to understand the world. This explains why small differences are invisible to beginners, but stark to experts. It also offers a hopeful message to the rest of us: all abilities are skills, if you practice them, your brain will get the message and devote more resources to them.

1.13.3. End Note

Münte, T. F., Altenmüller, E., & Jäncke, L. (2002). The musician's brain as a model for neuroplasticity. Nature Neuroscience Reviews, 3, 473-478. (This is a review paper rather than an original research report.)

1.13.4. See Also

Pantev, C., Oostenveld, R., Engelien, A., Ross, B., Roberts, L. E., & Hoke, M. (1998). Increased auditory cortical representation in musicians. Nature, 392, 811-814.
Pleger B., Dinse, H. R., Ragert, P., Schwenkreis, P., Malin, J. P., & Tegenthoff, M. (2001). Shifts in cortical representations predict human discrimination improvement. Proceedings of the National Academy of Sciences of the USA, 98, 12255-12260.

Hack 11. Why People Don't Work Like Elevator Buttons

2008-06-02T05:40:00.000-07:00

More intense signals cause faster reaction times, but there are diminishing returns: as a stimulus grows in intensity, eventually the reaction speed can't get any better. The formula that relates intensity and reaction speed is Pieron's Law.

It's a common illusion that if you are in a hurry for the elevator you can make it come quicker by pressing the button harder. Or more often. Or all the buttons at once. It somehow feels as if it ought to work, although of course we know it doesn't. Either the elevator has heard you, or it hasn't. How loud you call doesn't make any difference to how long it'll take to arrive.

But then elevators aren't like people. People do respond quicker to more stimulation, even on the most fundamental level. We press the brake quicker for brighter stoplights, jump higher at louder bangs. And it's because we all do this that we all fall so easily into thinking that things, including elevators, should behave the same way.

1.12.1. In Action

Give someone this simple task: she must sit in front of a screen and press a button as quickly as she can as soon as she sees a light flash on. If people were like elevators, the time it takes to press the button wouldn't be affected by the brightness of the light or the number of lights.

But people aren't like elevators and we respond quicker to brighter lights; in fact, the relationship between the physical intensity of the light and the average speed of response follows a precise mathematical form. This form is captured by an equation called Pieron's Law. Pieron's Law says that the time to respond to a stimulus is related to the stimulus intensity by the formula:

Reaction Time
R₀ + kI^-b

Reaction Time is the time between the stimulus appearing and you responding. I is the physical intensity of the signal. R₀ is the minimum time for any response, the asymptotic value representing all the components of the reaction time that don't vary, such as the time for light to reach your eye. k and b are constants that vary depending on the exact setup and the particular person involved. But whatever the setup and whoever the person, graphically the equation .

1.12.2. How It Works

In fact, Pieron's Law holds for the brightness of light, the loudness of sound, and even the strength of taste.¹ It says something fundamental about how we process signals and make decisionsthe physical nature of a stimulus carries through the whole system to affect the nature of the response. We are not binary systems! The actual number of photons of light or the amplitude of the sound waves that triggers us to respond influences how we respond. In fact, as well as affecting response time, the physical intensity of the stimulus also affects response force as well (e.g., how hard we press the button).

A consequence of the form of Pieron's Law is that increases in speed are easy for low-intensity stimuli and get harder as the stimulus gains more intensity. It follows a log scale, like a lot of things in psychophysics. The converse is also true: for quick reaction times, it's easier to slow people down than to speed them up.

Pieron's Law probably results because of the fundamental way the decisions have to be made with uncertain information. Although it might be clear to you that the light is either there or not, that's only because your brain has done the work of removing the uncertainty for you. And on a neural level, everything is uncertain because neural signals always have noise in them.

So as you wait for light to appear, your neuronal decision-making hardware is inspecting noisy inputs and trying to decide if there is enough evidence to say "Yes, it's there!" Looking at it like this, your response time is the time to collect enough neural evidence that something has really appeared. This is why Pieron's Law applies; more intense stimuli provide more evidence, and the way in which they provide more evidence results in the equation shown earlier.

To see why, think of it like this: Pieron's Law is a way of saying that the response time improves but at a decreasing rate, as the intensity (i.e., the rate at which evidence accumulates) increases. Try this analogy: stimulus intensity is your daily wage and making a response is buying a $900 holiday. If you get paid $10 a day, it'll take 90 days to get the money for the holiday. If you get a raise of $5, you could afford the holiday in 60 days30 days sooner. If you got two $5 raises, you'd be able to afford the holiday in 45 daysonly 15 days sooner than how long it would take with just one $5 raise. The time until you can afford a holiday gets shorter as your wage goes up, but it gets shorter more slowly, and if you do the math it turns out to be an example of Pieron's Law.

1.12.3. End Note

Pins, D., & Bonnet, C. (1996). On the relation between stimulus intensity and processing time: Pieron's law and choice reaction time. Perception & Psychophysics, 58(3), 390-400.

1.12.4. See Also

Stafford, T., & Gurney, K. G. (in press). The role of response mechanisms in determining reaction time performance: Pieron's law revisited. Psychonomic Bulletin & Review (in press).
Luce, R. D. (1986). Response Times: Their Role in Inferring Elementary Mental Organisation. New York: Clarendon Press. An essential one stop for all you need to know about modeling reaction times.
Pieron, H. (1952). The Sensations: Their Functions, Processes and Mechanisms. London: Frederick Muller Ltd. The book in which Pieron first proposed his law.

Hack 10. Detect the Effect of Cognitive Function on Cerebral Blood Flow

2008-06-02T05:39:00.001-07:00

When you think really hard, your heart rate noticeably increases.

The brain requires approximately 20% of the oxygen in the body, even during times of rest. Like the other organs in our body, our brain needs more glucose, oxygen, and other essential nutrients as it takes on more work. Many of the scanning technologies that aim to measure aspects of brain function take advantage of this. Functional magnetic resonance imaging (fMRI) [Hack #4] benefits from the fact that oxygenated blood produces slightly different electromagnetic signals when exposed to strong magnetic fields than deoxygenated blood and that oxygenated blood is more concentrated in active brain areas. Positron emission tomography (PET) [Hack #3] involves being injected with weakly radioactive glucose and reading the subsequent signals from the most active, glucose-hungry areas of the brain.

A technology called transcranial Doppler sonography takes a different approach and measures blood flow through veins and arteries. It takes advantage of the fact that the pitch of reflected ultrasound will be altered in proportion to the rate of flow and has been used to measure moment-to-moment changes in blood supply to the brain. It has been found to be particularly useful in making comparisons between different mental tasks. However, even without transcranial Doppler sonography, you can measure the effect of increased brain activity on blood flow by measuring the pulse.

1.11.1. In Action

For this exercise you will need to get someone to measure your carotid pulse, taken from either side of the front of the neck, just below the angle of the jaw. It is important that only very light pressure be useda couple of fingertips pressed lightly to the neck, next to the windpipe, should enable your friend to feel your pulse with little trouble.

First you need to take a measure of a resting pulse. Sit down and relax for a few minutes. When you are calm, ask your friend to count your pulse for 60 seconds. During this time, close your eyes and try to empty your mind.

With a baseline established, ask your friend to measure your pulse for a second time, using exactly the same method. This time, however, try and think of as many species of animals as you can. Keeping still and with your eyes closed, think hard, and if you get stuck, try thinking up a new strategy to give you some more ideas.

During the second session, your pulse rate is likely to increase as your brain requires more glucose and oxygen to complete its task. Just how much increase you'll see varies from person to person.

1.11.2. How It Works

Thinking of as many animals as possible is a type of verbal fluency task, testing how easily you can come up with words. To complete the task successfully, you needed to be able to coordinate various cognitive skills, for example, searching your memory for category examples, generating and using strategies to think up more names (perhaps you thought about walking through the jungle or animals from your local area) and checking you were not repeating yourself.

Neuropsychologists often use this task to test the executive system, the notional system that allows us to coordinate mental tasks to solve problems and work toward a goal, skills that you were using to think up examples of animals. After brain injury (particularly to the frontal cortex), this system can break down, and the verbal fluency task can be one of the tests used to assess the function of this system.

Research using PET scanning has shown similar verbal fluency tasks use a significant amount of brain resources and large areas of the cortex, particularly the frontal, temporal, and parietal areas.¹

Interestingly, in this study people who did best used less blood glucose than people who did not perform as well. You can examine this relationship yourself by trying the earlier exercise on a number of people. Do the people who do best show a slightly lower pulse than others? In these cases, high performers seem to be using their brain more efficiently, rather than simply using more brain resources.

Although measuring the carotid pulse is a fairly crude measure of brain activity compared to PET scanning, it is still a good indirect measure of brain activity for this type of high-demand mental task, as the carotid arteries supply both the middle and anterior cerebral arteries. They supply blood to most major parts of the cortex, including the frontal, temporal, parietal, and occipital areas, and so would be important in supplying the needed glucose and oxygen as your brain kicks into gear.

One problem with PET scanning is that, although it can localize activity to certain brain areas, it has poor temporal resolution, meaning it is not very good at detecting quick changes in the rate of blood flow. In contrast, transcranial Doppler sonography can detect differences in blood flow over very short periods of time (milliseconds). Frauenfelder and colleagues used this technique to measure blood flow through the middle and anterior cerebral arteries while participants were completing tasks that are known to need similar cognitive skills as the verbal fluency exercise.² They found that the rate of blood flow changed second by second, depending on exactly which part of the task the participant was tackling. While brain scanning can provide important information about which areas of the brain are involved in completing a mental activity, sometimes measuring something as simple as blood flow can fill in the missing pieces.

1.11.3. End Notes

Parks, R. W., Loewenstein, D. A., Dodrill, K. L., Barker, W. W., Yoshii, F., Chang, J. Y., Emran, A., Apicella, A., Sheramata, W. A., & Duara, R. (1988). Cerebral metabolic effects of a verbal fluency test: A PET scan study. Journal of Clinical and Experimental Neuropsychology, 10(5), 565-575.
Schuepbach, D., Merlo, M. C., Goenner, F., Staikov, I., Mattle, H. P., Dierks, T., & Brenner, H. D. (2002). Cerebral hemodynamic response induced by the Tower of Hanoi puzzle and the Wisconsin card sorting test. Neuropsychologia, 40(1), 39-53.

Hack 9. The Neuron

2008-06-02T05:38:00.002-07:00

There's a veritable electrical storm going on inside your head: 100 billion brain cells firing electrical signals at one another are responsible for your every thought and action.

A neuron, a.k.a. nerve cell or brain cell, is a specialized cell that sends an electrical impulse out along fibers connecting it, in turn, to other neurons. These guys are the wires of your very own personal circuitry.

What follows is a simplistic description of the general features of nerve cells, whether they are found sending signals from your senses to your brain, from your brain to your muscles, or to and from other nerve cells. It's this last class, the kind that people most likely mean when they say "neurons," that we are most interested in here. (All nerve cells, however, share a common basic design.)

Don't for a second think that the general structure we're describing here is the end of the story. The elegance and complexity of neuron design is staggering, a complex interplay of structure and noise; of electricity, chemistry, and biology; of spatial and dynamic interactions that result in the kind of information processing that cannot be defined using simple rules.¹ For just a glimpse at the complexity of neuron structure, you may want to start with this free chapter on nerve cells from the textbook Molecular Cell Biology by Harvey Lodish, Arnold Berk, Lawrence S. Zipursky, Paul Matsudaira, David Baltimore, and James Darnell and published by W. H. Freeman (http://www.ncbi.nlm.nih.gov/books/bv.fcgi?call=bv.View..ShowSection&rid=mcb.chapter.6074), but any advanced cell biology or neuroscience textbook will do to give you an idea of what you're missing here.

The neuron is made up of a cell body with long offshootsthese can be very long (the whole length of the neck, for some neurons in the giraffe, for example) or very short (i.e., reaching only to the neighboring cell, scant millimeters away). Signals pass only one way along a neuron. The offshoots receiving incoming transmissions are called dendrites. The outgoing end, which is typically longer, is called the axon. In most cases there's only one, long, axon, which branches at the tip as it connects to other neuronsup to 10,000 of them. The junction where the axon of one cell meets the dendrites of another is called the synapse. Chemicals, called neurotransmitters, are used to get the signal across the synaptic gap. Each neuron will release only one kind of neurotransmitter, although it may have receptors for many different kinds. The arrival of the electric signal at the end of the axon triggers the release of stores of the neurotransmitter that move across the gap (it's very small, after all) and bind to receptor sites on the other side, places on the neuron that are tuned to join with this specific type of chemical.

Whereas the signal between neurons uses neurotransmitters, internally it's electrical. The electrical signal is sent along the neuron in the form of an action potential.² This is what we mean when we say impulses, signals, spikes, or refer, in brain imaging speak, to the firing or lighting up of brain areas (because this is what activity looks like on the pictures that are made). Action potentials are the fundamental unit of information in the brain, the universal currency of the neural market.

The two most important computational features are as follows:

They are binary. A neuron either fires or doesn't, and each time it fires, the signal is the same size (there's more on this later). Binary signals stop the message from becoming diluted as neurons communicate with one another over distances that are massive compared to the molecular scale on which they operate.
Neurons encode information in the rate at which they send signals, not in the size of the signals they send. The signals are always the same size, information encoded in the frequency at which signals are sent. A stronger signal is indicated by a higher frequency of spikes, not larger single spikes. This is called rate coding.

Together these two features mean that the real language of the brain is not just a matter of spikes (signals sent by neurons), but spikes in time.

Whether or not a new spike, or impulse, is generated by the postsynaptic neuron (the one on the receiving side of the synapse) is affected by the following interwoven factors:

The amount of neurotransmitter released
The interaction with other neurotransmitters released by other neurons
How near they are and how close together in space and time
In what order they release their neurotransmitters

All of this short-term information is affected by any previous history of interaction between these two neuronstimes one has caused the other to fire and when they have both fired at the same time for independent reasonsand slightly adjusts the probability of interaction happening again.³

Spikes happen pretty often: up to once every 2 milliseconds at the maximum rate of the fastest-firing cells (in the auditory system; see Chapter 4 for more on that). Although the average rate of firing is responsive to the information being represented and transmitted in the brain, the actual timing of individual spikes is unpredictable. The brain seems to have evolved an internal communication system that has noise added to only one aspect of the information it transmitsthe timing, but not the size of the signals transmitted. Noise is a property of any biological system, so it's not surprising that it persists even in our most complex organ. It could very well also be the case that the noise [Hack #33] is playing some useful role in the information processing the brain does.

After the neurotransmitter has carried (or not carried, as the case may be) the signal across the synaptic gap, it's then broken down by specialized enzymes and reabsorbed to be released again when the next signal comes along. Many drugs work by affecting the rate and quantity of particular neurotransmitters released and the speed at which they are broken down and reabsorbed.

Hacks such as [Hack #11] and [Hack #26] show some of the other consequences for psychology of using neurons to do the work. Two good introductions to how neurons combine on a large scale can be found at http://www.foresight.gov.uk/cognitive.html. This is a British government Department of Trade and Industry project that aimed to get neuroscientists and computer scientists to collaborate in producing reviews of recent advances in their fields and summarize the implications for the development of artificial cognitive systems.

1.10.1. End Notes

Gurney, K. N. (2001). Information processing in dendrites II. Information theoretic complexity. Neural Networks, 14, 1005-1022.
You can start finding out details of the delicate electrochemical dance that allows the transmission of these binary electrical signals on the pages about action potentials that are part of a series of lecture notes on human physiology (http://members.aol.com/Bio50/LecNotes/lecnot11.html), the Neuroscience for Kids site (http://faculty.washington.edu/chudler/ap.html), and The Brain from Top to Bottom project (http://www.thebrain.mcgill.ca/flash/a/a_01/a_01_m/a_01_m_fon/a_01_m_fon.html).
But this is another storya story called learning.

1.10.2. See Also

How neurons are born, develop, and die is another interesting story and one that we're not covering here. These notes from the National Institutes of Health are a good introduction: http://www.ninds.nih.gov/health_and_medical/pubs/NINDS_Neuron.htm.
Neurons actually make up less than a tenth of the cells in the brain. The other 90-98%, by number, are glial cells, which are involved in development and maintenancethe sysadmins of the brain. Recent research also suggests that they play more of a role in information processing than was previously thought. You can read about this in the cover story from the April 2004 edition of Scientific American (volume 290 #4), "The Other Half of the Brain."

Hack 8. Tour the Cortex and the Four Lobes

2008-06-02T05:38:00.001-07:00

The forebrain, the classic image of the brain we know from pictures, is the part of the brain that defines human uniqueness. It consists of four lobes and a thin layer on the surface called the cortex.

When you look at pictures of the human brain, the main thing you see is the rounded, wrinkled bulk of the brain. This is the cerebrum, and it caps off the rest of the brain and central nervous system [Hack #7].

To find your way around the cerebrum, you need to know only a few things. It's divided into two hemispheres, left and right. It's also divided into four lobes (large areas demarcated by particularly deep wrinkles). The wrinkles you can see on the outside are actually folds: the cerebrum is a very large folded-up surface, which is why it's so deep. Unfolded, this surfacethe cerebral cortexwould be about 1.5 m² (a square roughly 50 inches on the side), and between 2 and 4 mm deep. It's not thick, but there's a lot of it and this is where all the work takes place. The outermost part, the top of the surface, is gray matter, the actual neurons themselves. Under a few layers of these is the white matter, the fibers connecting the neurons together. The cortex is special because it's mainly where our high-level, human functions take place. It's here that information is integrated and combined from the other regions of the brain and used to modulate more basic functions elsewhere in the brain. The folds exist to allow many more neurons and connections than other animals have in a similar size area.

1.9.1. Cerebral Lobes

The four cerebral lobes generally perform certain classes of function.

You can cover the frontal lobe if you put your palms on your forehead with your fingers pointing up. It's heavily involved in planning, socializing, language, and general control and supervision of the rest of the brain.

The parietal lobe is at the top and back of your head, and if you lock your fingers together and hook your hands over the top back, that's it covered there. It deals a lot with your senses, combining information and representing your body and movements. The object recognition module for visual processing [Hack #13] is located here.

You can put your hands on only the ends of the temporal lobeit's right behind the ears. It sits behind the frontal lobe and underneath the parietal lobe and curls up the underside of the cerebrum. Unsurprisingly, auditory processing occurs here. It deals with language too (like verbal memory), and the left hemisphere is specialized for this (non-linguistic sound is on the right). The curled-up ends of the temporal lobe join into the limbic system at the hippocampus and are involved in long-term memory formation.

Finally, there's the occipital lobe, right at the back of the brain, about midway down your head. This is the smallest lobe of the cerebrum and is where the visual cortex is located.

The two hemispheres are joined together by another structure buried underneath the lobes, called the corpus callosum. It's the largest bundle of nerve fibers in the whole nervous system. While sensory information, such as vision, is divided across the two hemispheres of the brain, the corpus callosum brings the sides back together. It's heavily coated in a fatty substance called myelin, which speeds electrical conduction along nerve cells and is so efficient that the two sides of the visual cortex (for example) operate together almost as if they're adjacent. Not bad considering the corpus callosum is connecting together brain areas a few inches apart when the cells are usually separated by only a millimeter or two.

1.9.2. Cerebral Cortex

The cortex, the surface of these lobes, is divided into areas performing different functions. This isn't exact, of course, and they're highly interconnected and draw information from one another, but more or less there are small areas of the surface that perform edge detection for visual information or detect tools as opposed to animate objects in much higher-level areas of the brain.

How these areas are identified is covered in the various brain imaging and methods hacks earlier in this chapter.

The sensory areas of the cortex are characterized by maps, representations of the information that comes in from the senses. It's called a map because continous variations in the value of inputs are represented by continuous shifts in distance between where they are processed in the cortical space. In the visual cortex, visual space is preserved on the retina. This spatial map is retained for each stage of early visual processing. This means that if two things are next to each other out there in the world they will, at least initially, be processed by contiguous areas of the visual cortex. This is just like when a visual image is stored on photographic negative but unlike when a visual image is stored in a JPEG image file. You can't automatically point to two adjoining parts of the JPEG file and be certain that they will appear next to each other in the image. With a photographic film and with the visual cortex, you can. Similarly, the auditory cortex creates maps of what you're hearing, but as well as organizing things according to where they appear in space, it also has maps that use frequency of the sound as the coordinate frame (i.e., they are tonotopic). And there's an actual map in physical space, on the cortex, of the whole body surface too, called the sensory homunculus [Hack #12] . You can tell how much importance the brain gives to areas of the map, comparatively, by looking at how large they are. The middle of the map of the primary visual cortex corresponds with the fovea in the retina, which is extremely high resolution. It's as large as the rest of the visual map put together.

When the cortex is discussed, that means the function in question is highly integrated with the rest of the brain. When we consider what really makes us human and where consciousness is, it isn't solely the cortex: the rest of the brain has changed function in humans, we have human bodies and nervous systems, and we exist within environments that our brains reflect in their adaptations. But it's definitely mostly the cortex. You are here.

Hack 7. Get Acquainted with the Central Nervous System

2008-06-02T05:36:00.002-07:00

Take a brief tour around the spinal cord and brain. What's where, and what does what?

Think of the central nervous system like a mushroom with the spinal cord as the stalk and the brain as the cap. Most of the hacks in this book arise from features in the cortex, the highly interconnected cells that make a thin layer over the brain...but not all. So let's start outside the brain itself and work back in.

Senses and muscles all over the body are connected to nerves, bundles of neurons that carry signals back and forth. Neurons come in many types, but they're basically the same wherever they're found in the body; they carry electric current and can act as relays, passing on information from one neuron to the next. That's how information is carried from the sensory surface of the skin, as electric signals, and also how muscles are told to move, by information going the other way.

Nerves at this point run to the spinal cord two by two. One of each pair of nerves is for receptors (a sense of touch for instance) and one for effectorsthese trigger actions in muscles and glands. At the spinal cord, there's no real intelligence yet but already some decision-makingsuch as the withdrawal reflexoccurs. Urgent signals, like a strong sense of heat, can trigger an effector response (such as moving a muscle) before that signal even reaches the brain.

The spinal cord acts as a conduit for nerve impulses up and down the body: sensory impulses travel up to the brain, and the motor areas of the brain send signals back down again. Inside the cord, the signals converge into 31 pairs of nerves (sensory and motor again), and eventually, at the top of the neck, these meet the brain.

At about the level of your mouth, right in the center of your head, the bundles of neurons in the spinal cord meet the brain proper. This tip of the spinal cord, called the brain stem, continues like a thick carrot up to the direct center of your brain, at about the same height as your eyes.

This, with some other central regions, is known as the hindbrain. Working outward from the brain stem, the other large parts of the brain are the cerebellum, which runs behind the soft area you can feel at the lower back of your head, and the forebrain, which is almost all the rest and includes the cortex.

Hindbrain activities are mostly automatic: breathing, the heartbeat, and the regulation of the blood supply.

The cerebellum is old brainalmost as if it were evolution's first go at performing higher-brain functions, coordinating the senses and movement. It plays an important role in learning and also in motor control: removing the cerebellum produces characteristic jerky movements. The cerebellum takes input from the eyes and ears, as well as the balance system, and sends motor signals to the brain stem.

Sitting atop the hindbrain is the midbrain, which is small in humans but much larger in animals like bats. For bats, this corresponds to a relay station for auditory informationbats make extensive use of their ears. For us, the midbrain acts as a connection layer, penetrating deep into the forebrain (where our higher-level functions are) and connecting back to the brain stem. It acts partially to control movement, linking parts of the higher brain to motor neurons and partially as a hub for some of the nerves that don't travel up the spinal cord but instead come directly into the brain: eye movement is one such function.

Now we're almost at the end of our journey. The forebrain, also known as the cerebrum, is the bulbous mass divided into two great hemispheresit's the distinctive image of the brain that we all know. Buried in the cerebrum, right in the middle where it surrounds the tip of the brain stem and midbrain, there's the limbic system and other primitive systems. The limbic system is involved in essential and automatic responses like emotions, and includes the very tip of the temporal cortex, the hippocampus and the amygdala, and, by some reckonings, the hypothalamus. In some animals, like reptiles, this is all there is of the forebrain. For them, it's a sophisticated olfactory system: smell is analyzed here, and behavioral responses like feeding and fighting are triggered.

Neuroscientist joke: the hypothalamus regulates the four essential Fs of life: fighting, fleeing, feeding, and mating.

T.S.

For us humans, the limbic system has been repurposed. It still deals with smell, but the hippocampus, for exampleone part of the systemis now heavily involved in long-term memory and learning. And there are still routing systems that take sensory input (from everywhere but the nose, which is routed directly to the limbic system), and distribute it all over the forebrain. Signals can come in from the rest of the cerebrum and activate or modulate limbic system processing common to all animalsthings like emotional arousal. The difference, for us humans, is that the rest of the cerebrum is so large. The cap of the mushroom consists of four large lobes on each hemisphere, visible when you look at the picture of the brain. Taken together, they make up 90% of the weight of the brain. And spread like a folded blanket over the whole of it is the layer of massively interconnected neurons that is the cerebral cortex, and if any development can be said to be responsible for the distinctiveness of humanity, this is it. For more on what functions the cerebral cortex performs, read [Hack #8].

As an orienting guide, it's useful to have a little of the jargon as well as the map of the central nervous system. Described earlier are the regions of the brain based mainly on how they grow and what the brain looks like. There are also functional descriptions, like the visual system [Hack #13], that cross all these regions. They're mainly self-explanatory, as long as you remember that functions tend to be both regions in the brain and pathways that connect areas together.

There are also positional descriptions, which describe the brain geographically and seem confusing on first encounter. They're often used, so it's handy to have a crib.

These terms are used to describe direction within the brain and prefix the Latin names of the particular region they're used with (e.g., posterior occipital cortex means the back of the occipital cortex).

Unfortunately, a number of different schemes are used to name the subsections of the brain, and they don't always agree on where the boundaries of the different regions are. Analogous regions in different species may have different names. Different subdisciplines use different schemes and conventions too. A neuropsychologist might say "Broca's areas," while a neuroanatomist might say "Brodman areas 44, 45, and 46"but they are both referring to the same thing. "Cortex" is also "neocortex" is also "cerebrum." The analogous area in the rat is the forebrain. You get the picture. Add to this the fact that many regions have subdivisions (the somatosensory cortex is in the parietal lobe, which is in the neocortex, for example) and some subdivisions can be put by different people in different supercategories, and it can get very confusing.

1.8.1. See Also

Three excellent online resources for exploring neuroanatomy are Brain Info (http://www.med.harvard.edu/AANLIB/home.html), The Navigable Atlas of the Human Brain (http://www.msu.edu/~brains/humanatlas), and The Whole Brain Atlas (http://braininfo.rprc.washington.edu/mainmenu.html).
The Brain Museum (http://brainmuseum.org) houses lots of beautifully taken pictures of the brains from more than 175 different species.
BrainVoyager (http://www.brainvoyager.com), which makes software for processing fMRI data, is kind enough to provide a free program that lets you explore the brain in 3D.
Nolte, J. (1999). The Human Brain: An Introduction to Its Functional Anatomy.
Crossman, A. R., & Neary, D. (2000). Neuroanatomy: An Illustrated Colour Text.

Hack 6. Neuropsychology, the 10% Myth, and Why You Use All of Your Brain

2008-06-02T05:36:00.001-07:00

Neuropsychology is the study of what different parts of the brain do by studying people who no longer have those parts. As well as being the oldest technique of cognitive neuroscience, it refutes the oft-repeated myth that we only use 10% of our brains.

Of the many unscientific nuggets of wisdom about the brain that many people believe, the most common may be the "fact" that we use only 10% of our brains.

In a recent survey of people in Rio de Janeiro with at least a college education, approximately half stated that the 10% myth was true.¹ There is no reason to suppose the results of a similar survey conducted anywhere else in the world would be radically different. It's not surprising that a lot of people believe this myth, given how often it is claimed to be true. Its continued popularity has prompted one author to state that the myth has "a shelf life longer than lacquered Spam".²

Where does this rather popular belief come from?

It's hard to find out how the myth started. Some people say that something like it was said by Einstein, but there isn't any proof. The idea that we have lots of spare capacity is certainly true and fits with our aspirational culture, as well as with the Freudian notion that the mind is mostly unconscious. Indeed, the myth was being used to peddle self-help literature as early as 1929.³ The neatness and numerological potency of the 10% figure is a further factor in the endurance of the myth.

A.B.

Neuropsychology is the study of patients who have suffered brain damage and the psychological consequences of that brain damage. As well as being a vital source of information about which bits of the brain are involved in doing which things, neuropsychology also provides a neat refutation of the 10% myth: if we use only 10% of our brains, which bits would you be happy to lose? From neuropsychology, we know that losing any bit of the brain causes you to stop being able to do something or being able to do it so well. It's all being used, not just 10% of it.

Admittedly we aren't clear on exactly what each bit of the brain does, but that doesn't mean that you can do without 90% of it.

Neuropsychology has other uses aside from disproving unhelpful but popularly held trivia. By looking at which psychological functions remain after the loss of a certain brain region, we can tell what brain regions are and are not necessary for us to do different things. We can also see how functions group and divide by looking at whether they are always lost together or lost only in dissimilar cases of brain damage. Two of the famous early discoveries of neuropsychology are two distinct language processing regions in the brain. Broca's area (named after the neuropsychologist Paul Broca) is in the frontal lobe and supports understanding and producing structure in language. Those with damage to Broca's area speak in stilted, single words. Wernicke's area (on the junction between the temporal and parietal lobes and named after Carl Wernicke) supports producing and understanding the semantics of language. People with brain damage to Wernicke's area can produce grammatically correct sentences, but often with little or no meaning, an incomprehensible "word salad."

Another line of evidence against the 10% myth is brain imaging research [[Hack#2] through [Hack#4]], which has grown exponentially in the last couple of decades. Such techniques allow the increased blood flow to be measured in certain brain regions during the performance of cognitive tasks. While debate continues about the degree to which it is sensible to infer much about functional localization from imaging studies, one thing they make abundantly clear is that there are no areas of the brain that are "black holes"areas that never "light up" in response to some task or other. Indeed, the neurons that comprise the cortex of the brain are active to some degree all the time, even during sleep.

A third line of argument is that of evolutionary theory. The human brain is a very expensive organ, requiring approximately 20% of blood flow from the heart and a similar amount of available oxygen, despite accounting for only 2% of body weight. The evolutionary argument is straightforward: is it really plausible that such a demanding organ would be so inefficient as to have spare capacity 10 times greater than the areas being usefully employed?

Fourth, developmental studies indicate that neurons that are not employed early in life are likely never to recover and behave normally. For example, if the visual system is not provided with light and stimulation within a fairly narrow developmental window, the neurons atrophy and vision never develops. If the visual system is deprived of a specific kind of stimulation, such as vertical lines, it develops without any sensitivity to that kind of stimulus. Functions in other parts of the brain similarly rely on activation to develop normally. If there really were a large proportion of neurons that were not used but were instead lying in wait, likely they would be useless by puberty.

It can be seen, then, that the 10% myth simply doesn't stand up to critical thinking. Two factors complicate the picture slightly, however; both have been used to muddy the waters around the claim at some stage.

First, people who suffer hydrocephalus in childhood have been seen to have large "holes" in the middle of their brains and yet function normally (the holes are fluid-filled ventricles that are present in every brain but are greatly enlarged in hydrocephalus). This condition has been the focus of sensationalist television documentaries, the thrust of which is that we can get on perfectly well without much of our brains. Such claims are willfully misleadingwhat such examples actually show is the remarkable capacity of the brain to assign functioning to alternative areas if there are problems with the "standard" areas during a specific time-point in development. Such "neuronal plasticity," as it is known, is not seen following brain damage acquired in adulthood. As discussed earlier, development of the brain depends on activitythis same fact explains why hydrocephalitic brains can function normally and makes having an unused 90% extremely unlikely.

Second, there is actually a very disingenuous sense in which we do "use" only 10% of our brains. The glial cells of the brain outnumber the neurons by a factor of roughly 10 to 1. Glial cells play a supporting role to the neurons, which are the cells that carry the electrochemical signals of the brain. It is possible, therefore, to note that only approximately 10% of the cells of the cortex are directly involved in cognition.

This isn't what proponents of the 10% theory are referring to, however. Instead, the myth is almost always a claim about mind, not brain. The claim is analogous to arguing that we operate at only 10% of our potential (although "potential" is so immeasurable a thing, it is misleading from the start to throw precise percentages around).

Uri Geller makes explicit the "untapped potential" interpretation in the introduction to Uri Geller's Mind-Power Book:

Our minds are capable of remarkable, incredible feats, yet we don't use them to their full capacity. In fact, most of us only use about 10 per cent of our brains, if that. The other 90 per cent is full of untapped potential and undiscovered abilities, which means our minds are only operating in a very limited way instead of at full stretch.

The confusion between brain and mind blurs the issue, while lending the claim an air of scientific credibility because it talks about the physical brain rather than the unknowable mind.

But it's just not true that 90% of the brain's capacity is just sitting there unused. It is true that our brains adjust their function according to experience [Hack #12] good news for the patients studied by neuropsychology. Many of them recover some of the ability they have lost. It is also true that the brain can survive a surprisingly large amount of damage and still sort of work (compare pouring two pints of beer down your throat and two pints of beer into your computer's hard disk drive for an illustration of the brain's superior resistance to insults). But neither of these facts mean that you have exactly 90% of untapped potentialyou need all your brain's plasticity and resistance to insult to keep learning and functioning across your life span.

In summary, the 10% myth isn't true, but it does offer an intuitively seductive promise of the possibility of self-improvement. It has been around for at least 80 years, and despite having no basis in current scientific knowledge and being refuted by at least 150 years of neuropsychology, it is likely to exist for as long as people are keen to aspire to be something more than they are.

1.7.1. End Notes

Herculano-Houzel, S. (2002). Do you know your brain? A survey on public neuroscience literacy at the closing of the decade of the brain. The Neuroscientist 8, 98-110.
Radford, B. (1999). The ten-percent myth. Skeptical Inquirer. March-April (http://www.csicop.org/si/9903/ten-percent-myth.html).
You can read all about the 10% myth in Beyerstein, B. L. (1999), Whence cometh the myth that we only use 10% of our brains? In Della Sala (ed.), Mind MythsExploring Popular Assumptions About the Mind and Brain. New York: John Wiley and Sons, 4-24, at snopes.com (http://www.snopes.com/science/stats/10percnt.htm), and in these two online essays by Eric Chudler, "Do We Use Only 10% of Our Brain?" (http://faculty.washington.edu/chudler/tenper.html) and "Myths About the Brain: 10 Percent and Counting" (http://www.brainconnection.com/topics/?main=fa/brain-myth).

Hack 5. Transcranial Magnetic Stimulation: Turn On and Off Bits of the Brain

2008-06-02T05:35:00.001-07:00

Stimulate or suppress specific regions of the brain, then sit back and see what happens.

Transcranial magnetic stimulation (TMS) isn't an imaging technique like EEG [Hack 2] or fMRI [Hack #4], but it can be used along with them. TMS uses a magnetic pulse or oscillating magnetic fields to temporarily induce or suppress electrical activity in the brain. It doesn't require large machines, just a small device around the head, andso far as we knowit's harmless with no aftereffects.

Neurons communicate using electrical pulses, so being able to produce electrical activity artificially has its advantages. Selected regions can be excited or suppressed, causing hallucinations or partial blindness if some part of the visual cortex is being targeted. Both uses help discover what specific parts of the brain are for. If the subject experiences a muscle twitching, the TMS has probably stimulated some motor control neurons, and causing hallucinations at different points in the visual system can be used to discover the order of processing (it has been used to discover where vision is cut out during saccades [Hack #17], for example).

Preventing a region from responding is also useful: if shutting down neurons in a particular area of the cortex stops the subject from recognizing motion, that's a good clue as to the function of that area. This kind of discovery was possible before only by finding people with localized brain damage; now TMS allows more structured experiments to take place.

Coupled with brain imaging techniques, it's possible to see the brain's response to a magnetic pulse ripple through connected areas, revealing its structure.

1.6.1. Pros

Affects neural activity directly, rather than just measuring it.

1.6.2. Cons

Apparently harmless, although it's still early days.

1.6.3. See Also

"Savant For a Day" by Lawrence Osbourne (http://www.nytimes.com/2003/06/22/magazine/22SAVANT.html or http://www.cognitiveliberty.org/neuro/TMS_NYT.html, an alternative URL), an article in the New York Times, which describes Lawrence Osborne's experience of TMS, having higher-level functions of his brain suppressed, and a different type of intelligence exposed.

Hack 4. Functional Magnetic Resonance Imaging: The State of the Art

2008-06-02T05:34:00.002-07:00

fMRI produces high-resolution animations of the brain in action.

Functional magnetic resonance imaging (fMRI) is the king of brain imaging. Magnetic resonance imaging is noninvasive and has no known side effectsexcept, for some, claustrophobia. Having an MRI scan requires you to lie inside a large electromagnet in order to be exposed to the high magnetic field necessary. It's a bit like being slid inside a large white coffin. It gets pretty noisy too.

The magnetic field pushes the hydrogen atoms in your brain into a state in which they all "line up" and spin at the same frequency. A radio frequency pulse is applied at this exact frequency, making the molecules "resonate" and then emit radio waves as they lose energy and return to "normal." The signal emitted depends on what type of tissue the molecule is in. By recording these signals, a 3D map of the anatomy of the brain is built up.

MRI isn't a new technology (it's been possible since the '70s), but it's been applied to psychology with BOLD functional MRI (abbreviated to fMRI) only as recently as 1992. To obtain functional images of the brain, BOLD (blood oxygen level dependent) fMRI utilizes the fact that deoxygenated blood is magnetic (because of the iron in hemoglobin) and therefore makes the MRI image darker. When neurons become active, fresh blood washes away the deoxygenated blood in the precise regions of the brain that have been more active than usual.

While structural MRI can take a long time, fMRI can take a snapshot of activity over the whole brain every couple of seconds, and the resolution is still higher than with PET [Hack #3]. It can view activity in volumes of the brain only 2 mm across and build a whole map of the brain from that. For a particular experiment, a series of fMRI snapshots will be animated over a single high-resolution MRI scan, and experimenters can see in exactly which brain areas activity is taking place.

Much of the cognitive neuroscience research done now uses fMRI. It's a method that is still developing and improving, but already producing great results.

1.5.1. Pros

High spatial resolution and good enough time resolution to look at changing patterns of activity. While not able to look at the changing brain as easily as EEG [Hack #2], its far greater spatial resolution means fMRI is suitable for looking at which parts of the brain are active in the process of recalling a fact, for example, or seeing a face.

1.5.2. Cons

Bulky, highly magnetic, and very expensive machinery.
fMRI is still new. It's a complex technique requiring computing power and a highly skilled team with good knowledge both of physics and of the brain.

Hack 3. Positron Emission Tomography: Measuring Activity Indirectly with PET

2008-06-02T05:34:00.001-07:00

PET is a radioactivity-based technique to build a detailed 3D model of the brain and its activity.

Positron emission tomography (PET) is more invasive than any of the other imaging techniques. It requires getting a radioactive chemical into the bloodstream (by injection) and watching for where in the brain the radioactivity ends upthe "positron emission" of the name. The level of radioactivity is not dangerous, but this technique should not be used on the same person on a regular basis.

When neurons fire to send a signal to other neurons, they metabolize more energy. A few seconds later, fresh blood carrying more oxygen and glucose is carried to the region. Using a radioactive isotope of water, the amount of blood flow to each brain location can be monitored, and the active areas of the brain that require a lot of energy and therefore blood flow can be deduced.

1.4.1. Pros

A PET scan will produce a 3D model of brain activity.

1.4.2. Cons

Scans have to take place in bulky, expensive machinery, which contain the entire body.
PET requires injecting the subject with a radioactive chemical.
Although the resolution of images has improved over the last 30 years, PET still doesn't produce as fine detail as other techniques (it can see activity about 1 cm across).
PET isn't good for looking at how brain activity changes over time. A snapshot can take minutes to be assembled.

Hack 2. Electroencephalogram: Getting the Big Picture with EEGs

2008-06-02T05:33:00.000-07:00

EEGs give you an overall picture of the timing of brain activity but without any fine detail.

An electroencephalogram (EEG) produces a map of the electrical activity on the surface of the brain. Fortunately, the surface is often what we're interested in, as the cortexresponsible for our complex, high-level functionsis a thin sheet of cells on the brain's outer layer. Broadly, different areas contribute to different abilities, so one particular area might be associated with grammar, another with motion detection. Neurons send signals to one another using electrical impulses, so we can get a good measure of the activity of the neurons (how busy they are doing the work of processing) by measuring the electromagnetic field nearby. Electrodes outside the skull on the surface of the skin are close enough to take readings of these electromagnetic fields.

Small metal disks are evenly placed on the head, held on by a conducting gel. The range can vary from two to a hundred or so electrodes, all taking readings simultaneously. The output can be a simple graph of signals recorded at each electrode or visualised as a map of the brain with activity called out.

1.3.1. Pros

The EEG technique is well understood and has been in use for many decades. Patterns of electrical activity corresponding to different states are now well-known: sleep, epilepsy, or how the visual cortex responds when the eyes are in use. It is from EEG that we get the concepts of alpha, beta, and gamma waves, related to three kinds of characteristic oscillations in the signal.
Great time resolution. A reading of electrical activity can be taken every few milliseconds, so the brain's response to stimuli can be precisely plotted.
Relatively cheap. Home kits are readily available. OpenEEG (http://openeeg.sourceforge.net), EEG for the rest of us, is a project to develop low-cost EEG devices, both hardware and software.

1.3.2. Cons

Poor spatial resolution. You can take only as many readings in space as electrodes you attach (up to 100, although 40 is common). Even if you are recording from many locations, the electrical signals from the scalp don't give precise information on where they originate in the brain. You are getting only information from the surface of the skull and cannot perfectly infer what and where the brain activity was that generated the signals. In effect this means that it's useful for looking at overall activity or activity in regions no more precise than an inch or so across.

Hack 1. Find Out How the Brain Works Without Looking Inside

2008-06-02T05:32:00.000-07:00

How do you tell what's inside a black box without looking in it? This is the challenge the mind presents to cognitive psychology.

Cognitive psychology is the psychology of the basic mental processesthings like perception, attention, memory, language, decision-making. It asks the question, "What are the fundamental operations on which mind is based?"

The problem is, although you can measure what goes into someone's head (the input) and measure roughly what they do (the output), this doesn't tell you anything about what goes on in between. It's a black box, a classic reverse engineering problem.¹ How can we figure out how it works without looking at the code?

These days, of course, we can use neuroimaging (like EEG [Hack 2], PET [Hack #3], and fMRI [Hack #4]) to look inside the head at the brain, or use information on anatomy and information from brain-damaged individuals [Hack #6] to inform how we think the brain runs the algorithms that make up the mind. But this kind of work hasn't always been possible, and it's never been easy or cheap. Experimental psychologists have spent more than a hundred years refining methods for getting insight into how the mind works without messing with the insides, and these days we call this cognitive psychology.

There's an example of a cognitive psychology-style solution in another book from the hacks series, Google Hacks (http://www.oreilly.com/catalog/googlehks). Google obviously doesn't give access to the algorithms that run its searches, so the authors of Google Hacks, Tara Calishain and Rael Dornfest, were forced to do a little experimentation to try and work it out. Obviously, if you put in two words, Google returns pages that feature both words. But does the order matter? Here's an experiment. Search Google for "reverse engineering" and then search for "engineering reverse." The results are different; in fact, they are sometimes different even when searching for words that aren't normally taken together as some form of phrase. So we might conclude that order does make a difference; in some way, the Google search algorithm takes into account the order. If you try to whittle a search down to the right terms, something that returned only a couple of hits, perhaps over time you could figure out more exactly how the order mattered.

This is basically what cognitive psychology tries to do, reverse engineering the basic functions of the mind by manipulating the inputs and looking at the results. The inputs are often highly restricted situations in which people are asked to make judgments or responses in different kinds of situations. How many words from the list you learned yesterday can you still remember? How many red dots are there? Press a key when you see an X appear on the screen. That sort of thing. The speed at which they respond, the number of errors, or the patterns of recall or success tell us something about the information our cognitive processes use, and how they use it.

A few things make reverse engineering the brain harder than reverse engineering software, however.

Biological systems are often complex, sometimes even chaotic (in the technical sense). This means that there isn't necessarily a one-to-one correspondence in how a change in input affects output. In a logic-based or linear system, we can clearly see causes and effects. The mind, however, doesn't have this orderly mapping. Small things have big effects and sometime big changes in circumstance can produce little obvious difference in how we respond. Biological functionsincluding cognitionare often supported by multiple processes. This means they are robust to changes in just one supporting process, but it also means that they don't always respond how you would have thought when you try and influence them.

People also aren't consistent in the same way software or machines usually are. Two sources of variability are noise and learning. We don't automatically respond in the same way to the same stimulus every time. This sometimes happens for no apparent reason, and we call this randomness noise. But sometimes our responses change for a reason, not because of noise, and that's because the very act of responding first time around creates feedback that informs our response pattern for the next time (for example, when you get a new bike, you're cautious with your stopping distance at first, but each time you have to stop suddenly, you're better informed about how to handle the braking next time around). Almost all actions affect future processing, so psychologists make sure that if they are testing someone the test subject has either done the thing in question many times before, and hence stopped changing his response to it, or he has never done it before.

Another problem with trying to guess how the mind works is that you can't trust people when they offer their opinion on why they did something or how they did it. At the beginning of the twentieth century, psychology relied heavily on introspection and the confusion generated led to the movement that dominated psychology until the '70s: behaviorism. Behaviorism insisted that we treat only what we can reliably measure as part of psychology and excluded all reference to internal structures. In effect we were to pretend that psychology was just the study of how stimuli were linked to outputs. This made psychology much more rigorous experimentally (although some would argue less interesting). Psychology today recognizes the need to posit mind as more than simple stimulus-response matching, although cognitive psychologists retain the behaviorists' wariness of introspection. For cognitive psychologists, why you think you did something is just another bit of data, no more privileged than anything else they've measured, and no more likely to be right.²

Cognitive psychology takes us a long way. Many phenomena discovered by cognitive and experimental psychology are covered in this bookthings like the attentional blink [Hack #39] and state-dependent recall [Hack #87] . The rigor and precision of the methods developed by cognitive psychology are still vital, but now they can be used in tandem with methods that give insight into the underlying brain structure and processes that are supporting the phenomenon being investigated.

1.2.1. End Notes

Daniel Dennett has written a brief essay called "Cognitive Science as Reverse Engineering" (http://pp.kpnet.fi/seirioa/cdenn/cogscirv.htm) in which he discusses the philosophy of this approach to mind.
A psychologist called Daryl Bem formalized this in "self-perception theory." He said "Individuals come to know their own attitudes, emotions and internal states by inferring them from observations of their own behavior and circumstances in which they occur. When internal cues are weak, ambiguous, or uninterpretable, the individual is in the same position as the outside observer." Bem, D. J., "Self Perception Theory." In L. Berkowitz (ed.), Advances in Experimental Social Psychology, volume 6 (1972).

Inside the Brain

2008-06-02T05:31:00.000-07:00

1.1. Hacks 1-12

It's never entirely true to say, "This bit of the brain is solely responsible for function X." Take the visual system [Hack #13], for instance; it runs through many varied parts of the brain with no single area solely responsible for all of vision. Vision is made up of lots of different subfunctions, many of which will be compensated for if areas become unavailable. With some types of brain damage, it's possible to still be able to see, but not be able to figure out what's moving or maybe not be able to see what color things are.

What we can do is look at which parts of the brain are active while it is performing a particular taskanything from recognizing a face to playing the pianoand make some assertions. We can provide input and see what output we getthe black box approach to the study of mind. Or we can work from the outside in, figuring out which abilities people with certain types of damaged brains lack.

The latter, part of neuropsychology [Hack #6], is an important tool for psychologists. Small, isolated strokes can deactivate very specific brain regions, and also (though more rarely) accidents can damage small parts of the brain. Seeing what these people can no longer do in these pathological cases, provides good clues into the functions of those regions of the brain. Animal experimentation, purposely removing pieces of the brain to see what happens, is another.

These are, however, pathology-based methodsless invasive techniques are available. Careful experimentationmeasuring response types, reaction times, and response changes to certain stimuli over timeis one such alternative. That's cognitive psychology [Hack #1], the science of making deductions about the structure of the brain through reverse engineering from the outside. It has a distinguished history. More recently we've been able to go one step further. Pairing techniques from cognitive psychology with imaging methods and stimulation techniques [Hack#2] through [Hack#5], we can manipulate and look at the brain from the outside, without having to, say, remove the skull and pull a bit of the cerebrum out. These imaging methods are so important and referred to so much in the rest of this book, we've provided an overview and short explanation for some of the most common techniques in this chapter.

In order that the rest of the book make sense, after looking at the various neuroscience techniques, we take a short tour round the central nervous system [Hack #7], from the spine, to the brain [Hack #8], and then down to the individual neuron [Hack #9] itself. But what we're really interested in is how the biology manifests in everyday life. What does it really mean for our decision-making systems to be assembled from neurons rather than, well, silicon, like a computer? What it means is that we're not software running on hardware. The two are one and the same, the physical properties of our mental substrate continually leaking into everyday life: the telltale sign of our neurons is evident when we respond faster to brighter lights [Hack #11], and our biological roots show through when blood flow has to increase because we're thinking so hard [Hack #10] .

And finally take a gander at a picture of the body your brain thinks you have and get in touch with your inner sensory homunculus [Hack #12] .

Why Mind Hacks?

2008-06-02T05:29:00.002-07:00

The term "hacking" has a bad reputation in the media. They use it to refer to those who break into systems or wreak havoc with computers as their weapons. Among people who write code, though, the term "hack" refers to a "quick-and-dirty" solution to a problem, or a clever way to get something done. And the term "hacker" is taken very much as a compliment, referring to someone as being "creative," having the technical chops to get things done. The Hacks series is an attempt to reclaim the word, document the good ways people are hacking, and pass the hacker ethic of creative participation on to the uninitiated. Seeing how others approach systems and problems is often the quickest way to learn about a new technology.

The brain, like all hidden systems, is prime territory for curious hackers. Thanks to relatively recent developments in cognitive neuroscience, we're able to satisfy a little of that curiosity, making educated explanations for psychological effects rather than just pointing those effects out, throwing light on the internal workings of the brain.

Some of the hacks in this collection document the neat tricks the brain has used to get the job done. Looking at the brain from the outside like this, it's hard not to be impressed at the way it works. Other hacks point to quirks of our own minds that we can exploit in unexpected ways, and that's all part of learning our way round the wrinkles in this newly exposed technology.

Mind Hacks is for people who want to know a bit more about what's going on inside own heads and for people who are going to assemble the hacks in new ways, playing with the interface between ourselves and the world. It's wonderfully easy to get involved. We've all got brains, after all.

Restrictions on the Superuser

2008-06-02T05:29:00.001-07:00

Because the superuser account is occasionally compromised—for example, by somebody sharing the superuser password with a friend—there have been numerous attempts to limit the availability and the power of the Unix superuser account.

5.4.1 Secure Terminals: Limiting Where the Superuser Can Log In

Most versions of Unix allow you to configure certain terminals so that users can't log in as the superuser from the login: prompt. Anyone who wishes to have superuser privileges must first log in as himself and then su to root. This feature makes tracking who is using the root account easier because the su command logs the username of the person who runs it and the time that it was run.^[13] Unix also requires that the root user's password be provided when booting in single-user mode if the console is not listed as being secure.

^[13] Unless you configure your syslog system so that this log is kept on a remote machine, the person who uses the su command can delete the logfile after successfully becoming root. For information on configuring the syslog system, see Chapter 21.

Secure consoles add to overall system security because they force people to know two passwords to gain superuser access to the system. Network virtual terminals should not be listed as secure to prevent users from logging into the root account remotely using telnet. (Of course, telnet should also be disabled, which it isn't in some environments.) The Secure Shell server ignores the terminal security attribute, but it has its own directive (PermitRootLogin in sshd_config) that controls whether users may log in as root remotely.

On BSD-derived systems, terminal security is specified in the /etc/ttys file. In this excerpt from the file, the tty00 terminal is secure and the tty01 terminal is not:

tty00   "/usr/libexec/getty std.9600"   unknown on secure
tty01   "/usr/libexec/getty std.9600"   unknown on

On System V-derived systems, terminal security is specified in the file /etc/securetty. This file specifies that tty1 and tty2 are secure:

# more /etc/securetty 
tty1
tty2
#

In general, most Unix systems today are configured so that the superuser can log in with the root account on the system console, but not on other terminals.

Even if your system allows users to log directly into the root account, we recommend that you institute rules that require users to first log into their own accounts and then use the su command.

5.4.2 BSD Kernel Security Levels

FreeBSD, Mac OS X, and other operating systems have kernel security levels, which can be used to significantly reduce the power that the system allots to the root user. Using kernel security levels, you can decrease the chances that an attacker who gains root access to your computer will be able to hide this fact in your logfiles.

The kernel security level starts at 0; it can be raised as part of the system startup, but never lowered. The secure level is set with the sysctl command:

sysctl kern.securelevel=1

Level 1 is used for secure mode. Level 2 is used for "very secure" mode. Level 3 is defined as the "really-really secure mode."

At security level 1, the following restrictions are in place:

Write access to the raw disk partitions is prohibited. (This forces all changes to the disk to go through the filesystem.)
Raw access to the SCSI bus controller is prohibited.
Files that have the immutable flag set cannot be changed. Files that have the append-only bit set can only be appended to, and not otherwise modified or deleted.
The contents of IP packets cannot be logged.
Raw I/O to the system console is prohibited.
Raw writes to system memory or I/O device controllers from user programs are prohibited.
Some access is denied to the Linux /proc filesystem.
Additional kernel modules cannot be loaded.
The system clock cannot be set backwards. In addition, it cannot be set forward more than a maximum of one second, and it can be set forward only once per second (effectively, the clock can be pushed at most to double time).

At security level 2, the following restriction is added:

Reads from raw disk partitions are not permitted.

At security level 3, the following restriction is added:

Changes to the IP filter are not permitted.

This list is not comprehensive.

Overall, setting the secure level to 1 or 2 enables you to increase the overall security of a Unix system; it also makes the system dramatically harder to administer. If you need to take an action that's prohibited by the current security level, you must reboot the system to do so. Furthermore, the restrictions placed on the root user at higher secure levels may not be sufficient; it may be possible, given enough persistence, for a determined attacker to circumvent the extra security that the secure level system provides. In this regard, setting the level higher may create a false sense of security that lulls the administrator into failing to put in the proper safeguards. Nevertheless, if you can run your system at a secure level higher than 0 without needing to constantly reboot it, it's probably worthwhile to do so.

5.4.3 Linux Capabilities

Another mechanism for limiting the power of the superuser is the Linux capabilities system, invented on other operating systems five decades ago and included with the Linux 2.4 kernel. Some other high-security Unix systems and security add-ons to Unix have used capabilities for years, and the POSIX committee drafted a standard (POSIX 1003.1e) but later withdrew it.

The Linux capabilities system allows certain privileged tasks to be restricted to processes that have a specific "capability." This capability can be used, transferred to other processes, or given up. Once a process gives up a capability, it cannot regain that capability unless it gets a copy of the capability from another process that was similarly endowed. At startup, the init process generates all of the capabilities that the operating system requires for its use. As processes start their operations, they shed unneeded capabilities. In this manner, compromising one of these processes does not compromise other aspects of the operating system, even if the compromised process is running as root.

Alternatives to the Superuser

Other operating systems—including Multics—obviate the superuser flaw by compartmentalizing the many system privileges that Unix bestows on the root user. Indeed, attempts to design a "secure" Unix (one that meets U.S. Government definitions of highly trusted systems) have adopted this same strategy of dividing superuser privileges into many different categories.

Unfortunately, attempts at compartmentalization often fail. For example, Digital's VAX/VMS operating system divided system privileges into many different classifications. But many of these privileges could be used by a persistent person to establish the others. For example, an attacker who achieves "physical I/O access" can modify the operating system's database to grant himself any other privilege that he desires. Thus, instead of a single catastrophic failure in security, we have a cascading series of smaller failures leading to the same end result. For compartmentalization to be successful, it must be carefully thought out.

Some of the capabilities that a program can give up in the Linux 2.4.19 kernel are shown in Table 5-2. (This table also provides a nice illustration of the power of the superuser!)

Table 5-2. Some capabilities in Linux 2.4.19
Capability	Description
CAP_CHOWN	Can change file owner and group
CAP_FOWNER	Can override file restrictions based on file owner ID
CAP_FSETIDCAP_SETUIDCAP_SETGID	Can override requirements for setting SUID and SGID bits on files
CAP_KILL	Can send signals to any process
CAP_LINUX_IMMUTABLE	Can change the immutable or append-only attributes on files
CAP_NET_BIND_SERVICE	Can bind to TCP/UDP ports below 1024
CAP_NET_BROADCAST	Can transmit broadcasts
CAP_NET_ADMIN	Can configure interfaces, bind addresses, modify routing tables and packet filters, and otherwise manage networking
CAP_NET_RAW	Can use raw and packet sockets
CAP_IPC_LOCK	Can lock shared memory
CAP_IPC_OWNER	Can override IPC ownership checks
CAP_SYS_MODULE	Can load and remove kernel modules
CAP_SYS_CHROOT	Can use chroot( )
CAP_SYS_PTRACE	Can ptrace( ) any process
CAP_SYS_PACCT	Can enable, disable, or configure process accounting
CAP_SYS_ADMIN	Can configure disk quotas, configure kernel logging, set hostnames, mount and unmount filesystems, enable and disable swap, tune disk devices, access system bios, set up serial ports, and many other things
CAP_SYS_BOOT	Can use reboot( )
CAP_SYS_NICE	Can change process priorities and scheduling
CAP_SYS_RESOURCE	Can set or override limits on resources, quotas, reserved filesystem space, and other things
CAP_SYS_TIME	Can manipulate system clock
CAP_SYS_TTY_CONFIG	Can configure tty devices
CAP_SETPCAP	Can transfer or remove capabilities from any other process

Unfortunately, at the time this edition is being written, few Linux systems are designed to take advantage of the kernel capabilities and few system programs have been written to shed capabilities.

The su Command: Changing Who You Claim to Be

2008-06-02T05:27:00.000-07:00

Sometimes, one user must assume the identity of another. For example, you might sit down at a friend's terminal and want access to one of your protected files. Rather than forcing you to log your friend out and log yourself in, Unix gives you a way to change your user ID temporarily: the su command, which is short for "substitute user." The su command requires that you provide the password of the user to whom you are changing.

For example, to change yourself from tim to john, you might type:

% whoami
tim
% /bin/su john
password: fuzbaby

% whoami
john
%

You can now access john's files. (And you will be unable to access tim's files, unless those files are specifically available to the user john.)

The most common use of the su command is to invoke superuser access. For example, if you are the system administrator and Rachel needs her password reset, you could reset the password by becoming the superuser and then using the passwd command:

$ /bin/su
Password: rates34
# passwd rachel
Changing local password for rachel.
New password:mymy5544
Retype new password:mymy5544
passwd: updating the database...
passwd: done
# exit
%

This will be discussed at length in Section 5.3.2.

5.3.1 Real and Effective UIDs with the su Command

Processes on Unix systems always have at least two identities. Normally, these two identities are the same. The first identity is the real UID. The real UID is your "real identity" and matches up (usually) with the username you logged in as. Sometimes you may want to take on the identity of another user to access some files or execute some commands. You might do this by logging in as that user, thus obtaining a new command interpreter whose underlying process has a real UID equal to that user.

Alternatively, if you only want to execute a few commands as another user, you can use the su command (as described in the previous section) to create a new process. This will run a new copy of your command interpreter (shell), and have the identity (real UID) of that other user. To use the su command, you must either know the password for the other user's account or be currently running as the superuser.

There are times when a software author wants a single command to execute with the rights and privileges of another user—most often, the root user. In a case such as this, we certainly don't want to disclose the password to the root account, nor do we want the user to have access to a command interpreter running as root. Unix addresses this problem through the use of a special kind of file designation called setuid or SUID. When a SUID file is run, the process involved takes on an effective UID that is the same as the owner of the file, but the real UID remains the same. SUID files are explained in Chapter 6.

5.3.1.1 Saved IDs

Some versions of Unix have a third form of UID: the saved UID. In these systems, a user may run a setuid program that sets an effective UID of 0 and then sets some different real UID as well. The saved UID is used by the system to allow the user to set her identity back to the original value. Normally, this is not something the user can see, but it can be important when you are writing or running SUID programs.

5.3.1.2 Other IDs

Unix also has the analogous concepts of effective GID , real GID, and setgid for groups.

Some versions of Unix also have concepts of session ID, process group ID, and audit ID. A session ID is associated with the processes connected to a terminal, and can be thought of as indicating a "login session." A process group ID designates a group of processes that are in the foreground or background on systems that allow job control. An audit ID indicates a thread of activity that should be treated as the same in the audit mechanism. You need to understand session IDs and process group IDs if you are developing software that needs to remain running after a user logs out, or if you are creating a system of programs that need to communicate with each other by using signals. Audit IDs are important if you are developing software that needs to analyze audit log files.

5.3.2 Becoming the Superuser

Typing su without a username tells Unix that you wish to become the superuser. You will be prompted for a password. Typing the correct root password causes a shell to be run with a UID of 0. When you become the superuser, your prompt should change to the pound sign (#) to remind you of your new powers. For example:

% /bin/su -
password: k697dgf
# whoami
root
#

Once you have become the superuser, you are free to perform whatever system administration you wish.

When using the su command to become the superuser, you should always type the command's full pathname, /bin/su. By typing the full pathname, you are assuring the system that you are actually running the real /bin/su command, and not another command named su that happens to be in your search path. This method is a very important way of protecting yourself (and the superuser password) from capture by a Trojan horse. Other techniques are described in Chapter 23.

Notice the use of the dash in the earlier example. Most versions of the susu to invoke its subshell with a dash, which causes the shell to read all relevant startup files and simulate a login. Using the dash option is important when becoming a superuser: the option guarantees that you will be using the superuser's path, and not the path of the account from which you sued. command support an optional argument of a single dash. When supplied, this causes

To exit the subshell, type exit.

If you use the su command to change to another user while you are the superuser, you won't be prompted for the password of that user. (This makes sense; as the superuser, you could easily change that user's password and then log in as that user.) For example:

# /bin/su john
% whoami
john
%

Using su to become the superuser is not a security hole. Any user who knows the superuser password could also log in as the superuser; breaking in through su is no easier. In fact, su enhances security: many Unix systems can be set up so that every su attempt is logged, with the date, time, and user who typed the command. Examining these log files allows the system administrator to see who is exercising superuser privileges—as well as who shouldn't be!

5.3.3 Use su with Caution

If you are the system administrator, you should be careful about how you use the su command. Remember that if you su to the superuser account, you can do things by accident that you would normally be protected from doing. You could also accidentally give away access to the superuser account without knowing you did so.

As an example of the first case, consider the real instance of someone we know who thought that he was in a temporary directory in his own account and typed rm -rf *. Unfortunately, he was actually in the /usr/lib directory, and he was operating as the superuser. He spent the next few hours restoring tapes, checking permissions, and trying to soothe irate users. The moral of this small vignette, and hundreds more we could relate with similar consequences, is that you should not issue commands as the superuser unless you need the extra privileges. Program construction, testing, and personal "housecleaning" should all be done under your own user identity.^[10]

^[10] Another good moral of this story is that you should always type rm -rf with a full pathname (e.g., rm -rf /usr/tmp/*—especially when running the command as the superuser!

Another example is when you accidentally execute a Trojan Horse program instead of the system command you thought you executed. (See the sidebar Stealing Superuser, later in this chapter.) If something like this happens to you as user root, your entire system can be compromised. We discuss some defenses to this in Chapter 23, but one major suggestion is worth repeating: if you need access to someone else's files, su to that user ID and access them as that user rather than as the superuser.

For instance, if a user reports a problem with files in her account, you could su to the root account and investigate, because you might not be able to access her account or files from your own, regular account. However, a better approach is to su to the superuser account, and then su to the user's account—you won't need her password for the su after you are root. Not only does this method protect the root account, but you will also have some of the same access permissions as the user you are helping, and that may help you find the problem sooner.

Stealing Superuser

Once upon a time, many years ago, one of us needed access to the root account on an academic machine. Although we had been authorized by management to have rootroot account was dangerous (correct), that he had far more knowledge of Unix than we did (unlikely), and that we didn't need the access (incorrect). After several diplomatic and bureaucratic attempts to get access normally, we took a slightly different approach, with management's wry approval. access, the local system manager didn't want to disclose the password. He asserted that access to the

We noticed that this user had "." at the beginning of his shell search path. This meant that every time he typed a command name, the shell would first search the current directory for the command of the same name. When he did a su to root, this search path was inherited by the new shell. This was all we really needed.

First, we created an executable shell file named ls in the current directory:

#!/bin/sh
cp /bin/sh ./stuff/junk/.superdude
chmod 4555 ./stuff/junk/.superdude
rm -f $0
exec /bin/ls ${1+"$@"}

Then, we executed the following commands:

% cd
% chmod 700 .
% touch ./-f

The trap was ready. We approached the recalcitrant administrator with the complaint, "I have a funny file in my directory I can't seem to delete." Because the directory was mode 700, he couldn't list the directory to see the contents. So, he used su to become user root. Then he changed the directory to our home directory and issued the command ls to view the problem file. Instead of the system version of ls, he ran our version. This created a hidden setuid root copy of the shell, deleted the bogus lsls command. The administrator never knew what happened. command, and ran the real

We listened politely as he explained (superciliously) that files beginning with a dash character (-) needed to be deleted with a pathname relative to the current directory (in our case, rm ./-f); of course, we knew that.

A few minutes later, he couldn't get the new root password.

5.3.4 Using su to Run Commands from Scripts

Another common use of the su command is to run a program under a specific userID in a script that is being run automatically by root. For example, a startup script for a system that runs three programs under three different user IDs might look like this:

/bin/su usera -c /usr/local/system/scripta
/bin/su userb -c /usr/local/system/scriptb
/bin/su userc -c /usr/local/system/scriptc

Early versions of the Unix cron program ran all programs in the crontab under the user root; to run a program under a different user, the su command was used:

0 4 * * * /bin/su uucp -c /usr/lib/uucp/uuclean

5.3.5 Restricting su

On some versions of Berkeley-derived Unix, a user cannot su to the root account unless the user is a member of the Unix group wheel—or any other group given the group ID of 0. For this restriction to work, the /etc/group entry for group wheelsu to user root if he has the password. must be non-empty; if the entry has no usernames listed, the restriction is disabled, and anyone can

Some versions of su also allow members of the wheel group to become the superuser by providing their own passwords instead of the superuser password. The advantage of this feature is that you don't need to tell the superuser's password to a user for him to have superuser access—you simply have to put him into the wheel group. You can take away his access simply by taking him out of the group.

Some versions of System V Unix require that users specifically be given permission to su. Different versions of Unix accomplish this in different ways; consult your own system's documentation for details, and use the mechanism if it is available.

Another way to restrict the su program is by making it executable only by a specific group and by placing in that group only the people who you want to be able to run the command. For information on how to do this, see Section 6.3.

5.3.6 The su Log

Most versions of the su command log successful and failed attempts. Older versions of Unix explicitly logged su attempts to the console and to a hardcoded file, such as the /var/adm/messages file. Newer versions log bad su attempts through the syslog facility, allowing you to send the messages to a file of your choice or to log facilities on remote computers across the network. The FreeBSD version of su uses the syslog facility, but opens the facility with the LOG_CONS flag so that the bad su attempts are logged both to the auth facility and to the console. You should be careful who has access to the log of failed su attempts, as the log files can occasionally contain a variation of the root password.

If you notice many bad attempts, it may be an indication that somebody using an account on your system is trying to gain unauthorized privileges; this might be a legitimate user poking around, or it might be an indication that the user's account has been appropriated by an outsider who is trying to gain further access.

A single bad attempt, of course, might simply be a mistyped password, someone mistyping the du command, or somebody wondering what the su command does.^[11]

^[11] Which of course leads us to observe that people who try commands to see what they do shouldn't be allowed to run commands like su once they find out.

5.3.6.1 The sulog under Solaris

You can quickly scan the appropriate su log file for bad passwords with the grep command:

% grep BAD /var/adm/messages
BADSU 09/12 18:40 - pts/0 rachel-root

Good su attempts on a Solaris system look like this:

% grep + /var/adm/sulog
SU 09/14 23:42 + pts/2 simsong-root
SU 09/16 08:40 + pts/4 simsong-root
SU 09/16 10:34 + pts/3 simsong-root

It would appear that Simson has been busy suing to root on September 14th and 16th.

5.3.6.2 The sulog under Berkeley Unix

Here is a similar command executed on a FreeBSD system:

r2# grep 'su:' /var/log/messages
Jun 14 19:22:25  r2 su: simsong to root on /dev/ttyp1
Jun 14 19:30:06  r2 su: BAD SU simsong to root on /dev/ttyp1
Jun 14 19:30:18  r2 su: BAD SU simsong to root on /dev/ttyp1
Jun 14 19:31:10  r2 su: BAD SU simsong to root on /dev/ttyp2
Jun 14 19:31:38  r2 su: simsong to root on /dev/ttyp2
r2#

Note that the successful su attempts are logged with the syslog level , while the failed attempts are logged at the level . For more information on syslog warning levels, see Chapter 20.

5.3.6.3 The sulog under Red Hat Linux

Red Hat uses the pam_unix module to log su attempts to the /var/log/messagessu attempts look like this: file. Successful

# grep 'su.pam_unix' messages | grep -v failure
Jun 11 04:05:59 l1 su(pam_unix)[19838]: session opened for user news by (uid=0)
Jun 11 04:06:00 l1 su(pam_unix)[19838]: session closed for user news
Jun 11 15:48:37 l1 su(pam_unix)[22433]: session opened for user root by
simsong(uid=500)
Jun 11 15:51:23 l1 su(pam_unix)[22433]: session closed for user root
Jun 11 16:31:16 l1 su(pam_unix)[22695]: session opened for user root by
simsong(uid=500)
Jun 11 19:06:03 l1 su(pam_unix)[22695]: session closed for user root
#

Note that the pam_unix system logs successful su attempts by both users and programs. The pam_unix system also logs when the su session starts and ends. In the preceding example, the first two lines represent the start and end of a Netnews cleanup script that is run automatically at 4:00 a.m. every day by the operating system. UID 0 (the superuser) successfully sus to the news user, and then runs a script. The second and third attempts represent interactive su attempts by the user simsong.

Failed su attempts are logged to the same file, but with a different error message:

# grep 'su.pam_unix' messages | grep failure
Jun 15 14:40:55 l1 su(pam_unix)[10788]: authentication failure; logname=rachel
uid=181 euid=0 tty= ruser= rhost=  user=root
Jun 15 14:40:59 l1 su(pam_unix)[10789]: authentication failure; logname=rachel
uid=181 euid=0 tty= ruser= rhost=  user=root
#

These two examples indicate that user rachel attempted to su to the root account, and failed.

5.3.6.4 Final caution

The root account is not an account designed for the personal use of the system administrator. Because all security checks are turned off for the superuser, a typing error could easily trash the entire system. Murphy's Law ensures that this happens more often than even experienced users might wish, so use the root account with caution!

5.3.7 sudo: A More Restrictive su

Mac OS X, OpenBSD, and many Linux distributions are equipped with a program named sudo that allows a person to exercise superuser privileges on a single command. The commands executed as superuser are logged with the name of the person who has run the command, and the time that the command was executed. Security of logs can be increased if they are stored on a second computer; sudosudo attempt fails. can also send email messages when it runs successfully, or when a

To be allowed to use the sudo command, the user must be listed in the file sudoers, which is usually found in /etc or /usr/local/etc. The sudo command can be configured to allow users to use their own passwords or special sudo passwords.

The sudo command offers accountability. For example, on a Mac OS X computer, you'll see:

[G3:/var/log] simsong% sudo passwd rachel
Password: rates34
Changing local password for rachel.
New password:mymy5544
Retype new password:mymy5544
passwd: updating the database...
passwd: done
[G3:/var/log] simsong%

This results in the following entry being saved in the system's logfile:

Jun 11 16:36:38 G3 sudo:  simsong : TTY=ttyp1 ; PWD=/Users/simsong ; USER=root ;
COMMAND=/usr/bin/passwd

Another advantage of sudo is that the /etc/sudoers file can specify not only who may use sudo, but which commands they are permitted to run.^[12] For example, simsong may be allowed to run only passwd, dump, or mount.

^[12] In fact, it can even specify on which machines the users may run the commands. This makes it possible for a single sudoers file to be distributed across many machines.

It's important to be careful about which commands you allow users to run through sudo. Many commands, such as editors, provide a way to escape to a shell. If a user can run an editor as root, they can often escape to a root shell:

[G3:/var/log] simsong% sudo ed /dev/null
Password: rates34
0
!sh
[G3:/var/log] root#

At this point, the user has full access to the system and can run any command without having it logged.

The Superuser (root)

2008-06-02T05:25:00.000-07:00

Almost every Unix system comes with a special user in the /etc/passwd file with a UID of 0. This user is known as the superuser and is normally given the username root. The password for the root account is usually called simply the "root password."

The root account is the identity used by the operating system itself to accomplish its basic functions, such as logging users in and out of the system, recording accounting information, and managing input/output devices. For this reason, the superuser exerts nearly complete control over the operating system: nearly all security restrictions are bypassed for any program that is run by the root user, and most of the checks and warnings are turned off.^[6]

^[6] On a few systems, it's possible to restrict root's capabilities as part of the kernel boot process, so that even if the superuser account is compromised, some kinds of damage are not possible unless the attacker is physically at the console and has an additional password. Systems that use MAC often do not have a superuser at all, so the discussion in this section does not apply to such systems.

5.2.1 What the Superuser Can Do

Any process that has an effective UID of 0 (see Section 5.3.1 later in this chapter) runs as the superuser—that is, any process with a UID of 0 runs without security checks and is allowed to do almost anything. Normal security checks and constraints are ignored for the superuser, although most systems do audit and log some of the superuser's actions.

Some of the things that the superuser can do include:

Process control

Change the nice value of any process (see Section B.1.3.3).
Send any signal to any process (see Signals).
Alter "hard limits" for maximum CPU time as well as maximum file, data segment, stack segment, and core file sizes (see Chapter 23).
Turn accounting and auditing on and off (see Chapter 21).
Bypass login restrictions prior to shutdown. (Note that this may not be possible if you have configured your system so that the superuser cannot log into terminals.)
Change his process UID to that of any other user on the system.
Log out all users and prevent new logins.

Device control

Access any working device.
Shut down or reboot the computer.
Set the date and time.
Read or modify any memory location.
Create new devices (anywhere in the filesystem) with the mknod command.

Network control

Run network services on "trusted" ports (see Chapter 17).
Reconfigure the network.
Put the network interface into "promiscuous mode" and examine all packets on the network (possible only with certain kinds of networks and network interfaces).

Filesystem control

Read, modify, or delete any file or program on the system (see Chapter 6).
Run any program.^[7]

^[7] If a program has a file mode of 000, root must set the execute bit of the program with the chmod( ) system call before the program can be run, although shell scripts can be run by feeding their input directly into /bin/sh.
Change a disk's electronic label.^[8]

^[8] Usually stored on the first 16 blocks of a hard disk or floppy disk formatted with the Unix filesystem.
Mount and unmount filesystems.
Add, remove, or change user accounts.
Enable or disable quotas and accounting.
Use the chroot( ) system call, which changes a process's view of the filesystem root directory.
Write to the disk after it is "100 percent" full. The Berkeley Fast Filesystem and the Linux ext2 File System both allow the reservation of some minfree amount of the disk. Normally, a report that a disk is 100% full implies that there is still 10% left. Although this space can be used by the superuser, it shouldn't be: filesystems run faster when their disks are not completely filled.

5.2.2 What the Superuser Can't Do

Despite all of the powers listed in the previous section, there are some things that the superuser can't do, including:

Make a change to a filesystem that is mounted read-only. (However, the superuser can make changes directly to the raw device, or can unmount a read-only filesystem and remount it read/write, provided that the media is not physically write-protected.)
Unmount a filesystem that contains open files, or one in which some running process has set its current directory.^[9]

^[9] Many BSD variants (including NetBSD and FreeBSD) provide an -f option to umount, which forcibly unmounts a busy filesystem.
Write directly to a directory, or create a hard link to a directory (although these operations are allowed on some Unix systems).
Decrypt the passwords stored in the shadow password file, although the superuser can modify the /bin/login and su system programs to record passwords when they are typed. The superuser can also use the passwd command to change the password of any account.
Terminate a process that has entered a wait state inside the kernel, although the superuser can shut down the computer, effectively killing all processes.

5.2.3 Any Username Can Be a Superuser

As we noted in the next art., any account that has a UID of 0 has superuser privileges. The username root is merely a convention. Thus, in the following sample /etc/passwd file, both root and beth can execute commands without any security checks:

root:x:0:1:Operator:/:/bin/ksh
beth:x:0:101:Beth Cousineau:/u/beth:/bin/csh
rachel:x:181:181:Rachel Cohen:/u/rachel:/bin/ksh

You should immediately be suspicious of accounts on your system that have a UID of 0 that you did not install; accounts such as these are frequently added by people who break into computers so that they will have a simple way of obtaining superuser access in the future.

5.2.4 The Problem with the Superuser

The superuser is the main security weakness in the Unix operating system. Because the superuser can do anything, after a person gains superuser privileges—for example, by learning the root password and logging in as root—that person can do virtually anything to the system. This explains why most attackers who break into Unix systems try to become the superuser.

Most Unix security holes that have been discovered are of the kind that allow regular users to obtain superuser privileges. Thus, most Unix security holes result in a catastrophic bypass of the operating system's security mechanisms. After a flaw is discovered and exploited, the entire computer is compromised.

There are a number of techniques for minimizing the impact of such system compromises, including:

Storing sensitive files on removable media, and mounting the media only when you need to access the files. An attacker who gains superuser privileges while the media are unmounted will not have access to critical files.
Encrypting your files. Being the superuser grants privileges only on the Unix system; it does not magically grant the mathematical prowess necessary to decrypt a well-coded file or the necessary clairvoyance to divine encryption keys. (Encryption is discussed in C 7.) Best practice is to encrypt with a passphrase other than your login password, which an attacker might capture.
Mounting disks read-only when possible.
Taking advantage of filesystem features like immutable and append-only files if your system supports them.

Users and Groups

2008-06-02T05:23:00.000-07:00

Although every Unix user has a username consisting of one or more characters, inside the computer Unix represents the identity of each user by a single number: the user identifier (UID). Under most circumstances, each user is assigned his own unique ID.

Unix also uses special usernames for a variety of system functions. As with usernames associated with human users, system usernames usually have their own UIDs as well. Here are some common "users" on various versions of Unix:

root: Superuser account. Performs accounting and low-level system functions.
bin: Binary owner. Has ownership of system files on some systems but doesn't typically execute programs.
daemon: Handles some aspects of the network. This username is also associated with other utility systems, such as the print spoolers, on some versions of Unix.
mail: Handles aspects of electronic mail. On many systems there is no mail user, and daemon is used instead.
guest: Used (infrequently) for site visitors to access the system.
ftp: Used for anonymous FTP access.
uucp: Controls ownership of the Unix serial ports. (uucp traditionally managed the UUCP system, which is now deprecated.)
news: Used for Usenet news.
lp: Used for the printer system.^[1]

^[1] lp stands for line printer, although these days most people seem to be using laser printers.
nobody: Owns no files and is sometimes used as a default user for unprivileged operations.
www or http: Runs the web server.
named: Runs the BIND name server.
sshd: Performs unprivileged operations for the OpenSSH Secure Shell daemon.
operator: Used for creating backups and (sometimes) for printer operation.
games: Allowed to access high-score files.
amanda: Used for the Amanda remote backup system.

5.1.1 The /etc/passwd File

On most Unix systems the user accounts are listed in the database file /etc/passwd; the corresponding passwords for these accounts are kept in a file named /etc/shadow, /etc/security/passwd, or /etc/master.passwd. To improve lookup speed, some systems compile the password file into a compact index file named something like /etc/pwd.db, which is used instead.

Here is an example of an /etc/passwd file from a Linux system containing a variety of system and ordinary users:

$ more /etc/passwd
root:x:0:0:Mr. Root:/root:/bin/bash
bin:x:1:1:Binary Installation User:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
news:x:9:13:news:/var/spool/news:
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
gopher:x:13:30:gopher:/var/gopher:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
mailnull:x:47:47::/var/spool/mqueue:/dev/null
rpm:x:37:37::/var/lib/rpm:/bin/bash
xfs:x:43:43:X Font Server:/etc/X11/fs:/bin/false
ntp:x:38:38::/etc/ntp:/sbin/nologin
rpc:x:32:32:Portmapper RPC user:/:/bin/false
gdm:x:42:42::/var/gdm:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin
nscd:x:28:28:NSCD Daemon:/:/bin/false
ident:x:98:98:pident user:/:/sbin/nologin
rachel:x:181:181:Rachel Cohen:/u/rachel:/bin/ksh
ralph:x:182:182:Ralph Knox:/u/ralph:/bin/tcsh
mortimer:x:183:183:Mortimer Merkle:/u/mortimer:/bin/sh

Notice that most of these accounts do not have "people names," and that all have a password field of "x". In the old days of Unix, the second field was used to hold the user's encrypted password. This information is now stored in a second file, the shadow password file.

The /etc/passwd file can be thought of as a directory^[2] that lists all of the users on the system. As we saw in the last chapter, it is possible to configure a Unix system to use other directory services, such as NIS, NIS+, LDAP, and Kerberos. (We'll discuss directory services in detail in Chapter 14.) When these systems are used, the Unix operating system is often modified so that the utility programs still respond as if all of the accounts actually reside in a single /etc/passwd file.

^[2] Technically, it is a simple relational database.

5.1.2 User Identifiers (UIDs)

UIDs are historically unsigned 16-bit integers, which means they can range from 0 to 65535. UIDs between 0 and 99 are typically used for system functions; UIDs for humans usually begin at 100 or 1000. Many versions of Unix now support 32-bit UIDs. A few older versions of Unix have UIDs that are signed 16-bit integers, ranging from -32768 to 32767.

There is one special UID, which is UID 0. This is the UID that is reserved for the Unix superuser. The Unix kernel disables most security checks when a process is being run by a user with the UID of 0.

There is generally nothing special about any Unix account name. All Unix privileges are determined by the UID (and sometimes the group ID, or GID), and not directly by the account name. Thus, an account with name root and UID 1005 would have no special privileges, but an account named mortimer with UID 0 would be a superuser.

In general, you should avoid creating users with a UID of 0 other than root, and you should avoid using the name root for a regular user account. In this book, we will use the terms "root" and "superuser" interchangeably to mean a UID of 0.

Unix keeps the mapping between usernames and UIDs in the file /etc/passwd. Each user's UID is stored in the field after the one containing the user's encrypted password. For example, consider the sample /etc/passwd entry presented in Chapter 4:

rachel:x:181:181:Rachel Cohen:/u/rachel:/bin/ksh

In this example, Rachel's username is rachel and her UID is 181.

The UID is the actual information that the operating system uses to identify the user; usernames are provided merely as a convenience for humans. If two users are assigned the same UID, Unix views them as the same user, even if they have different usernames and passwords. Two users with the same UID can freely read and delete each other's files and can kill each other's running programs. Giving two users the same UID is almost always a bad idea; it is better to create multiple users and put them in the same group, as we will see later.

Conversely, files can be owned by a UID that is not listed in /etc/passwd as having an associated username. This is also a bad idea. If a user is added to /etc/passwd in the future with that UID, that user will suddenly become the owner of the files.

5.1.3 Groups and Group Identifiers (GIDs)

Every Unix user belongs to one or more groups. As with user accounts, groups have both a group name and a group identification number (GID). GID values are also historically 16-bit integers, but many systems now use 32-bit integers for these, too.

As the name implies, Unix groups are used to group users together. As with usernames, group names and numbers are assigned by the system administrator when each user's account is created. Groups can be used by the system administrator to designate sets of users who are allowed to read, write, and/or execute specific files, directories, or devices.

Each user belongs to a primary group that is stored in the /etc/passwd file. The GID of the user's primary group follows the user's UID. Historically, every Unix user was placed in the group users, which had a GID of 100. These days, however, most Unix sites place each account in its own group. This results in decreased sharing but somewhat greater security.^[3]

^[3] The advantage of assigning each user his own group is that it allows users to have a unified umask of 007 in all instances. When users wish to restrict access of a file or directory to themselves, they leave the group set to their individual group. When they wish to open the file or directory to members of their workgroup or project, all they need to do is to change the file's or directory's group accordingly.

Consider, again, our /etc/passwd example:

rachel:x:181:181:Rachel Cohen:/u/rachel:/bin/ksh

In this example, Rachel's primary GID is 181.

Groups provide a handy mechanism for treating a number of users in a certain way. For example, you might want to set up a group for a team of students working on a project so that students in the group, but nobody else, can read and modify the team's files.

Groups can also be used to restrict access to sensitive information or specially licensed applications to a particular set of users: for example, many Unix computers are set up so that only users who belong to the kmem group can examine the operating system's kernel memory. The operator group is commonly used to allow only specific users to run the tape backup system, which may have "read" access to the system's raw disk devices. And a sources group might be limited to people who have signed nondisclosure forms so they can view the source code for particular software.

Some special versions of Unix support mandatory access controls (MAC), which have controls based on data labeling instead of, or in addition to, the traditional Unix discretionary access controls (DAC). MAC-based systems do not use traditional Unix groups. Instead, the GID values and the /etc/group file may be used to specify security access control labeling or to point to capability lists. If you are using one of these systems, you should consult the vendor documentation to ascertain what the actual format and use of these values might be.

5.1.3.1 The /etc/group file

The /etc/group file contains the database that lists every group on your computer and its corresponding GID. Its format is similar to the format used by the /etc/passwd file.^[4]

^[4] As with the password file, if your site is running NIS, NIS+, or DCE, the /etc/group file may be incomplete or missing. See the discussion in Section 4.3.1 in Chapter 4.

Here is a sample /etc/group file that defines six groups: wheel, http, vision, startrek, rachel, and users:

wheel:*:0:root,rachel
http:*:10:http
users:*:100:
vision:*:101:keith,arlin,janice
startrek:*:102:janice,karen,arlin
rachel:*:181:

The first line of this file defines the wheel group. The fields are explained in Table 5-1.

Table 5-1. The first line of the example /etc/group file
Field contents	Description
`wheel`	Group name
*	Group's "password" (described later)
0	Group's GID
`root`, `rachel`	List of the users who are in the group

Most versions of Unix use the wheel group as the list of all of the computer's system administrators (in this case, rachel and the root user are the only members). On some systems, the group has a GID of 0; on other systems, the group has a GID of 10. Unlike a UID of 0, a GID of 0 is usually not significant. However, the name wheel is very significant: on many systems the use of the su command to invoke superuser privileges is restricted to users who are members of a group named wheel.

The second line of this file defines the http group. There is one member in the httphttp user. group—the

The third line defines the users group. The users group does not explicitly list any users; on some systems, each user is placed into this group by default through his individual entry in the /etc/passwd file.

The fourth and fifth lines define two groups of users. The vision group includes the users keith, arlin, and janice. The startrek group contains the users janice, karen, and arlin. Notice that the order in which the usernames are listed on each line is not important. (This group is depicted graphically in Figure 5-1.)

Finally, the sixth line defines a group for the user rachel.

Remember that the users mentioned in the /etc/group file are in these groups in addition to the groups mentioned as their primary groups in the file /etc/passwd. For example, Rachel is in the rachel group even though she does not appear in that group in the file /etc/group because her primary group number is 181. On most versions of Unix, you can use the groups command to list which groups that you are currently in:

% groups
rachel wheel
%

The groups command can also take a username as an argument:

% groups arlin
vision, startrek
%

When a user logs into the Unix system, the /bin/login program scans the /etc/passwd and /etc/group files, determines which groups the user is a member of, and adds them to the user's user structure using the setgroups( ) system call.^[5]

^[5] If you are on a system that uses NIS, NIS+, or some other system for managing user accounts throughout a network, these network databases will be referenced as well. For more information, see Chapter 19.

Some versions of Unix are equipped with an id command that offers more detailed UIDs, GIDs, and group lists:

% id
uid=181(rachel) gid=181(rachel) groups=181(rachel), 0(wheel)
% id root
uid=0(root) gid=0(wheel) groups=0(wheel),1(bin),15(shadow),65534(nogroup)

Figure 5-1 illustrates how users can be included in multiple groups.

Figure 5-1. Users and groups

Group Passwords

The newgrp command is used to change the user's active group. This is useful when a user wants to create files owned by a group other than his default group.

$ id
uid=1001(alansz) gid=20(users)
$ newgrp project
$ id
uid=1001(alansz) gid=100(project)

Solaris and other versions of Unix derived from AT&T SVR4 allow users to use newgrp to switch to a group that they are not a member of if the group is equipped with a group password:

$ newgrp fiction
password: rates34
$

The user is now free to exercise all of the rights and privileges of the fiction group instead of his default group.

The password in the /etc/group file is interpreted exactly like the passwords in the /etc/passwd file, including salts (described in Chapter 4 and Chapter 19). However, most systems do not have a program to install or change the passwords in this file. To set a group password, you must first assign it to a user with the passwd command, then use a text editor to copy the encrypted password out of the /etc/passwd file into the /etc/group file. Alternatively, you can encode the password using the /usr/lib/makekey program (if present) and edit the result into the /etc/group file in the appropriate place.

Group passwords are rarely used and can represent a security vulnerability, as an attacker can put a password on a critical group as a way of creating a back door for future access.

It is not necessary for there to be an entry in the /etc/group file for a group to exist! As with UIDs and account names, Unix actually uses only the integer part of the GID for all settings and permissions. The name in the /etc/group file is simply a convenience for the users—a means of associating a mnemonic with the GID value.

Pluggable Authentication Modules (PAM)

2008-06-02T05:22:00.002-07:00

Because there are so many ways to authenticate users, it's convenient to have a unified approach to authentication that can handle multiple authentication systems for different needs. The Pluggable Authentication Modules (PAM) system is one such approach.

PAM was originally developed by Sun, and implementations are available for Solaris, FreeBSD, and especially Linux, where most PAM development is now centered. PAM provides a library and API that any application can use to authenticate users against a myriad of authentication systems. Each authentication system that PAM knows about is implemented as a PAM module, which in turn is implemented as a dynamically-loaded shared library. PAM modules are available to authenticate users through:

/etc/passwd or /etc/shadow
NIS or NIS+
LDAP
Kerberos 4 or 5
An arbitrary Berkeley DB file^[20]

^[20] If that's not enough layers for you, some applications, such as SMTP authentication in sendmail or access to mailboxes managed by the Cyrus imapd server, use the Cyrus SASL (Simple Authentication and Security Layer) authentication library, which can authenticate users with a separate database or through PAM! It is not inconceivable that you might find SASL using PAM using LDAP to authenticate a user's IMAP connection.

Each PAM-aware service is configured either in the /etc/pam.conf file or, more commonly, in its own file in the /etc/pam.d directory. For example, the PAM configuration file for the /bin/su command in Linux distributions is /etc/pam.d/su. A service named other is used to provide defaults for PAM-aware services that are not explicitly configured.

Here is an excerpt from /etc/pam.conf for the OpenSSH server:

sshd  auth     required    /lib/security/pam_env.so
sshd  auth     sufficient  /lib/security/pam_unix.so likeauth nullok
sshd  auth     required    /lib/security/pam_deny.so
sshd  account  required    /lib/security/pam_unix.so

sshd password required   /lib/security/pam_cracklib.so retry=3
sshd password sufficient /lib/security/pam_unix.so nullok use_authtok md5 shadow
sshd password required   /lib/security/pam_deny.so

sshd  session  required  /lib/security/pam_limits.so
sshd  session  required  /lib/security/pam_unix.so

Here's how the same excerpt looks in /etc/pam.d/sshd:

auth     required   /lib/security/pam_env.so
auth     sufficient /lib/security/pam_unix.so
auth     required   /lib/security/pam_deny.so

account  required  /lib/security/pam_unix.so

password required  /lib/security/pam_cracklib.so retry=3
password sufficient /lib/security/pam_unix.so nullok use_authtok md5 shadow
password required   /lib/security/pam_deny.so

session  required   /lib/security/pam_limits.so
session  required   /lib/security/pam_unix.so

The auth lines describe the authentication process for this service, which proceeds in the order specified. Modules marked required must run successfully for authentication to progress—if they fail, the user is considered unauthenticated and generally will be denied access. Multiple required modules can be specified; in these cases, all of the modules must run successfully. Modules marked sufficient, if run successfully, are sufficient to authenticate the user and end the authentication process.

Note the modules in this example:

pam_env: The first module run, pam_env, optionally sets or clears environment variables specified in /etc/security/pam_env.conf. This module is required—it must run successfully for authentication to proceed.
pam_unix: The next module run, pam_unix, performs authentication with the usual Unix password files, /etc/passwd and /etc/shadow. If this succeeds, it is sufficient to authenticate the user, and the process is complete.
pam_deny: The final authentication module, pam_deny, simply fails, ending the process with authentication unsuccessful.

This particular configuration file will also enforce any account aging or expiration rules of the system, and set resources limits on the user's sshd session. If sshd provided a password-changing function, this configuration file would also prevent the user from changing his password to an easily guessable one, and store passwords in /etc/shadow encrypted by the MD5 cryptographic hash function.

The PAM subsystem can be configured in a number of different ways. For instance, it is possible to require two or three separate passwords for some accounts^[21] to combine a biometric method along with a passphrase, or to pick a different mechanism depending on the time of day. It is also possible to remove the requirement of a password for hardwired lines in highly secured physical locations. PAM allows the administrator to pick a policy that best matches the risk and technology at hand.

^[21] This is highly annoying and of questionable value when the same user holds all of the passwords. This approach can be valuable when the passwords are assigned to different users, so that any login requires two or more people and creates a "witness" trail.

PAM can do a lot more than authentication, as these examples suggest. One of its strengths is that it clearly delineates four phases of the access process.

Verifying that the account is viable for the desired service at the desired time and from the desired location (the account phase)
Authenticating the user (the auth phase)
Updating passwords and other authentication tokens when necessary (the password phase)
Setting up and closing down the user's session (the session phase), which can include limiting resource access and establishing audit trails

Network Account and Authorization Systems

2008-06-02T05:22:00.001-07:00

These days, many organizations have moved away from large time-sharing computers and invested in large client/server networks containing many servers and dozens or hundreds of workstations. These systems are usually set up so that any user can make use of any workstation in a group or in the entire organization. When these systems are in use, every user effectively has an account on every workstation. These systems provide for automatic account creation and password synchronization between some or many computer systems.

When you are working with a large, distributed system, it is not practical to ensure that every computer has the same /etc/passwd file. For this reason, there are now several different commercial systems available that make the information traditionally stored in the /etc/passwd file available over a network.

4.4.1 Using Network Authorization Systems

Five network authorization systems in use today are:

Sun Microsystems' Network Information System (NIS) and NIS+.
MIT Kerberos, which is now part of the OSF Distributed Computing Environment (DCE) and Microsoft's Windows XP. Kerberos clients are also included with Solaris, Linux, and several other Unix versions.
NetInfo, originally developed by NeXT Computer, now part of Mac OS X.
RADIUS, the Remote Authentication Dial-In User Service. Traditionally, RADIUS has been used by many ISPs to provide for authentication of dialup users. It has been extended to provide authentication for other devices (e.g., routers) and for password synchronization in a Unix environment.
Authentication systems that store account information in a Lightweight Directory Access Protocol (LDAP) server.

These systems all take the information that is usually stored in each workstation's /etc/passwd file and store it in one or more network servers. Some systems use the network-based account to supersede the accounts on the local system; others augment the local accounts with network-based accounts.

Some of these systems provide for multiple servers or backup caching, should the primary server be unavailable. Others do not, and instead create a single point of failure for the entire network.

At some sites, administrators prefer not to use network database management systems. Instead, each computer might have its own accounts. Alternatively, one computer might be regarded as the "master computer," and that computer's /etc/passwd and /etc/shadow files are then distributed to other computers using scp, rdist, or a similar system. There are several reasons that an administrator might make such a decision.

Managing a network-based authentication system is often considerably more complex than managing accounts on a single system.
Unless redundant servers are provided, a crashed authentication server or failed network segment can negatively impact a disproportionately large number of users.
The administrator might be fearful that the central authentication server could be compromised, which would allow an attacker to create an account on any computer that the attacker wished.

The drawback to this approach is that it often requires the administrator to intervene to change a user password or shell entry. In most cases, the energy spent developing and fielding custom solutions would be better spent mastering systems that are already in existence and, in many cases, preinstalled on most Unix systems.

Because there are so many different ways to access the information that has traditionally been stored in the /etc/passwd file, throughout this book we will simply use the phrase "password file" or "/etc/passwd " as a shorthand for the multitude of different systems.

4.4.2 Viewing Accounts in the Network Database

If you are using one of these systems and wish to retrieve the contents of the password database, you cannot simply cat the /etc/passwd file. Instead, you must use a command that is specific to your system to view the account database.

Sun's NIS service supplements the information stored in the workstations' own files. If you are using NIS and you wish to get a list of every user account, you would use the following command:

% cat /etc/passwd;ypcat passwd

4.4.2.1 NIS and NIS+

Sun's NIS+ service can be configured to supplement or substitute its user account entries for those entries in the /etc/passwd file, depending on the contents of the /etc/nsswitch.conf file. If you are using a system that runs NIS+, you should use the niscat command and specify your NIS+ domain. For example:

% niscat -o passwd.bigco

4.4.2.2 Kerboros DCE

Computers that are using DCE use an encrypted network database system as an alternative to encrypted passwords and /etc/passwd files. However, to maintain compatibility, some of them have programs that run on a regular basis to create a local /etc/passwd file. You should check your manuals for information about your specific system.

4.4.2.3 NetInfo

On Mac OS X systems running NetInfo, you can view the account database using the command:

% nidump passwd .

Note again that Mac OS X's system exposes the encrypted password field when the nidump command is used. Thus, although Mac OS X uses the FreeBSD master.passwd file, it still exposes the entire password database to anyone who wants it. This happens whether or not a network server is in use.

4.4.2.4 RADIUS

Systems that are configured for RADIUS generally do not make it possible to access the entire account database at once.

4.4.2.5 LDAP

LDAP is used to build a true network authentication system; rather than create local /etc/passwd entries, systems that use LDAP for authentication are configured to check logins against the network's LDAP server each time (though some configurations do include a name service-caching daemon^[19] [nscd] that caches LDAP responses locally to reduce the number of network authentications required). LDAP is covered in detail in Chapter 14.

^[19] Don't confuse this "name service" with Domain Name Service (DNS). Although nscd can cache DNS lookups of hostnames, its primary strength is its ability to cache lookups of users, groups, and passwords made through local files, NIS, NIS+, LDAP, and other authentication systems.

How Unix Implements Passwords

2008-06-02T05:21:00.001-07:00

This section describes how passwords are implemented inside the Unix operating system for both locally administered and network-based systems.

4.3.1 The /etc/passwd File

Traditionally, Unix uses the /etc/passwd file to keep track of every user on the system. The /etc/passwd file contains the username, real name, identification information, and basic account information for each user. Each line in the file contains a database record; the record fields are separated by a colon (:).

You can use the cat command to display your system's /etc/passwd file. Here are a few sample lines from a typical file:

root:x:0:1:System Operator:/:/bin/ksh
daemon:x:1:1::/tmp:
uucp:x:4:4::/var/spool/uucppublic:/usr/lib/uucp/uucico
rachel:x:181:100:Rachel Cohen:/u/rachel:/bin/ksh
arlin:x.:182:100:Arlin Steinberg:/u/arlin:/bin/csh

The first three accounts, root, daemon, and uucp, are system accounts, while rachel and arlin are accounts for individual users.

The individual fields of the /etc/passwd file have fairly straightforward meanings. Table 4-1 explains a sample line from the file shown above.

Table 4-1. Example /etc/passwd fields
Field	Contents
rachel	Username.
x	Holding place for the user's "encrypted password." Traditionally, this field actually stored the user's encrypted password. Modern Unix systems store encrypted passwords in a separate file (the shadow password file) that can be accessed only by privileged users.
181	User's user identification number (UID).
100	User's group identification number (GID).
Rachel Cohen	User's full name (also known as the GECOS or GCOS field).^[12]
/u/rachel	User's home directory.
/bin/ksh	User's shell.^[13]

^[12] When Unix was first written, it ran on a small minicomputer. Many users at Bell Labs used their Unix accounts to create batch jobs to be run via Remote Job Entry (RJE) on the bigger GECOS computer in the Labs. The user identification information for the RJE was kept in the /etc/passwd file as part of the standard user identification. GECOS stood for General Electric Computer Operating System; GE was one of several major companies that made computers around that time.

^[13] An empty field for the shell name does not mean that the user has no shell; instead, it means that a default shell—usually the Korn shell (/bin/ksh) or Bourne shell (/bin/sh)—should be used. To prevent a user from logging in, the program /bin/false is often used as the "shell."

Passwords were traditionally stored in the /etc/passwd file in an encrypted format (hence the file's name). However, because of advances in processor speed, encrypted passwords are now almost universally stored in separate shadow password files, which are described later.

The meanings of the UID and GID fields are described in Chapter 5.

4.3.2 The Unix Encrypted Password System

When Unix requests your password, it needs some way of determining that the password you type is the correct one. Many early computer systems (and quite a few still around today!) kept the passwords for all of their accounts plainly visible in a so-called "password file" that contained exactly that—passwords. Under normal circumstances, the system protected the passwords so that they could be accessed only by privileged users and operating system utilities. But through accident, programming error, or deliberate act, the contents of the password file could occasionally become available to unprivileged users. This scenario is illustrated in the following remembrance:

Perhaps the most memorable such occasion occurred in the early 1960s when a system administrator on the CTSS system at MIT was editing the password file and another system administrator was editing the daily message that is printed on everyone's terminal on login. Due to a software design error, the temporary editor files of the two users were interchanged and thus, for a time, the password file was printed on every terminal when it was logged in.

—Robert Morris and Ken Thompson, "Password Security: A Case History" Communications of the ACM, November 1979.

The real danger posed by such systems, explained Morris and Thompson, is not that software problems might someday cause a recurrence of this event, but that people can make copies of the password file and purloin them without the knowledge of the system administrator. For example, if the password file is saved on backup tapes, then those backups must be kept in a physically secure place. If a backup tape is stolen, then everybody's password needs to be changed.

Unix avoids this problem by not keeping actual passwords anywhere on the system. Instead, Unix stores a value that is generated by using the password to encrypt a block of zero bits with a one-way function called crypt( ); the result of the calculation was traditionally stored in the /etc/passwd file.^[14] When you try to log in, the program /bin/login does not decrypt the stored password. Instead, /bin/login takes the password that you typed, uses it to transform another block of zeros, and compares the newly transformed block with the block stored in the /etc/passwd file. If the two encrypted results match, the system lets you in.

^[14] These days, the encrypted password is stored either in the shadow password file or on a network-based server, as we'll see in a later section.

The security of this approach rests upon the strength of the encryption algorithm and the difficulty of guessing the user's password. To date, the crypt ( ) algorithm and its successors have proven highly resistant to attacks. Unfortunately, users have a habit of picking easy-to-guess passwords, which creates the need for shadow password files.

4.3.2.1 The traditional crypt ( ) algorithm

The algorithm that traditional crypt( ) uses is based on the Data Encryption Standard (DES) of the National Institute of Standards and Technology (NIST). In normal operation, DES uses a 56-bit key (8 7-bit ASCII characters, for instance) to encrypt blocks of original text, or cleartext, that are 64 bits in length. The resulting 64-bit blocks of encrypted text, or ciphertext, cannot easily be decrypted to the original cleartext without knowing the original 56-bit key.

The Unix crypt( ) function takes the user's password as the encryption key and uses it to encrypt a 64-bit block of zeros. The resulting 64-bit block of ciphertext is then encrypted again with the user's password; the process is repeated a total of 25 times. The final 64 bits are unpacked into a string of 11 printable characters that are stored in the shadow password file.^[15]

^[15] Each of the 11 characters holds six bits of the result, represented as one of 64 characters in the set ".", "/", 0-9, A-Z, a-z, in that order. Thus, the value 0 is represented as ".", and 32 is the letter "U".

Don't confuse the crypt( ) algorithm with the crypt encryption program. The crypt program uses a different encryption system from crypt( ) and is very easy to break. See Chapter 7 for more details.

Although the source code to crypt( ) is readily available, no technique has been discovered (or publicized) to translate the encrypted password back into the original password. Such reverse translation may not even be possible. As a result, the only known way to defeat Unix password security is via a brute-force attack (see the next note), or by a dictionary attack. A dictionary attack is conducted by choosing likely passwords—as from a dictionary—encrypting them, and comparing the results with the value stored in /etc/passwd. This approach to breaking a cryptographic cipher is also called a key search or password cracking. It is made easier by the fact that DE uses only the first eight characters of the password as its key; dictionaries need only contain passwords of eight characters or fewer.

Robert Morris and Ken Thompson designed crypt( ) to make a key search computationally expensive. The idea was to make a dictionary attack take too long to be practical. At the time, software implementations of DES were quite slow; iterating the encryption process 25 times made the process of encrypting a single password 25 times slower still. On the original PDP-11 processors upon which Unix was designed, nearly a full second of computer time was required to encrypt a single password. To eliminate the possibility of using DES hardware encryption chips, which were a thousand times faster than software running on a PDP-11, Morris and Thompson modified the DES tables used by their software implementation, rendering the two incompatible. The same modification also served to prevent a bad guy from simply pre-encrypting an entire dictionary and storing it.

What was the modification? Morris and Thompson added a bit of salt, as we'll describe in the next section.

There is no published or known method to easily decrypt DES-encrypted text without knowing the key. Of course, "easily" has a different meaning for cryptographers than for mere mortals. To decrypt something encrypted with DES is computationally expensive; using the fastest current, general-purpose computers might take hundreds of years.

However, computers have grown so much faster in the past 25 years that it is now possible to test millions of passwords in a relatively short amount of time.

4.3.2.2 Unix salt

As table salt adds zest to popcorn, the salt that Morris and Thompson sprinkled into the DES algorithm added a little more spice and variety. The DES salt is a 12-bit number, between 0 and 4,095, which slightly changes the result of the DES function. Each of the 4,096 different salts makes a password encrypt a different way.

When you change your password, the /bin/passwd program selects a salt based on the time of day. The salt is converted into a two-character string and is stored in the /etc/passwd file along with the encrypted "password."^[16] In this manner, when you type your password at login time, the same salt is used again. Unix stores the salt as the first two characters of the encrypted password.

^[16] By now, you know that what is stored in the /etc/passwd file is not really the encrypted password. However, everyone calls it that, and we will do the same from here on. Otherwise, we'll need to keep typing "the superencrypted block of zeros that is used to verify the user's password" everywhere in the book, filling many extra pages and contributing to the premature demise of yet more trees.

Table 4-2 shows how a few different words encrypt with different salts.

Table 4-2. Passwords and salts
Password	Salt	Encrypted password
`nutmeg`	Mi	MiqkFWCm1fNJI
`ellen1`	ri	ri79KNd7V6.Sk
`Sharon`	./	./2aN7ysff3qM
`norahs`	am	amfIADT2iqjAf
`norahs`	7a	7azfT5tIdyh0I

Notice that the last password, norahs, was encrypted two different ways with two different salts. As a side effect, the salt makes it possible for a user to have the same password on a number of different computers and to keep this fact a secret (usually), even from somebody who has access to the /etc/passwd files on all of those computers; two systems would not likely assign the same salt to the user, thus ensuring that the encrypted password field is different.^[17]

^[17] This case occurs only when the user actually types in his password on the second computer. Unfortunately, in practice, system administrators commonly cut and paste /etc/passwd entries from one computer to another when they build accounts for users on new computers. As a result, others can easily tell when a user has the same password on more than one system.

On the Importance of Encrypted Passwords

Alec Muffett, the author of the Crack program (discussed in Table 19-1), related an entertaining story to us about the reuse of passwords in more than one place, which we paraphrase here.

A student friend of Alec's (call him Bob) spent a co-op year at a major computer company site. During his vacations and on holidays, he'd come back to school and play AberMUD (a network-based game) on Alec's computer. One of Bob's responsibilities at the company involved system management. The company was concerned about security, so all passwords were random strings of letters with no sensible pattern or order.

One day, Alec fed the AberMUD passwords into his development version of the Crack program as a dictionary, because they were stored on his machine as plaintext. He then ran this file against his system user-password file, and found a few student account passwords. He had the students change their passwords, and he then forgot about the matter.

Some time later, Alec posted a revised version of the Crack program and associated files to the Usenet. They ended up in one of the Usenet sources newsgroups and were distributed quite widely. Eventually, after a trip of thousands of miles around the world, they came to Bob's company. Bob, being a concerned administrator, decided to download the files and check them against his company's passwords. Imagine Bob's shock and horror when the widely distributed Crack promptly churned out a match for his randomly chosen, super-secret root password!

The moral of the story is that you should teach your users never to use their account passwords for other purposes—such as games or web sites. They never know when those passwords might come back to haunt them! For developers, the moral is that all programs—even games—should store passwords encrypted with one-way hash functions.

In recent years the security provided by the salt has diminished significantly. Having a salt means that the same password can encrypt in 4,096 different ways. This makes it much harder for an attacker to build a reverse dictionary for translated encrypted passwords back into her unencrypted form: to build a reverse dictionary of 100,000 words, an attacker would need to have 409,600,000 entries. But with 8-character passwords and 13-character encrypted passwords, 409,600,000 entries fit in roughly 8 GBs of storage.

Another problem with the salt was an error in implementation: many systems selected which salt to use based on the time of day, which made some salts more likely than others.

4.3.2.3 crypt16( ), DES Extended, and Modular Crypt Format

Modern Unix systems have improved the security of the crypt( ) function by changing the underlying encryption algorithm. Instead of a modified DES, a variety of other algorithms have been adopted, including Blowfish and MD5. The advantage of these new algorithms is that more characters of the password are significant, and there are many more possible values for the salt; both of these changes significantly improve the strength of the underlying encrypted password system. The disadvantage is that the encrypted passwords on these systems will not be compatible with the encrypted passwords on other systems.

Because of the widespread use of the original Unix password encryption algorithm, Unix vendors have gone to great lengths to ensure compatibility. Thus, the crypt( ) function called with a traditional salt will always use the original DES-based algorithm. To use one of the newer algorithms you must use either a different function call (some vendors use bigcrypt( ) or crypt16( )) or a different salt value. Consult your documentation to find out what is appropriate for your system.

The DES Extended format is a technique for increasing the number of DES rounds and extending the salt from 2¹² to 2²⁴ possible values. This format has limited use on modern Unix systems but is included on many to provide backwards compatibility.

The Modular Crypt Format (MCF) specifies an extensible scheme for formatting encrypted passwords. MCF is one of the most popular formats for encrypted passwords around today. Here is an example of an MCF-encrypted password:

$1$EqkVUoQ2$4VLpJuZ.Q2wm6TAiyYt75.

Dollar signs are used to delimit the MCF fields, as described in Table 4-3.

Table 4-3. The modular crypt format
Field	Purpose	Notes
#1	Specifies encryption algorithm to use	1 specifies MD5.2 specifies Blowfish.
#2	Salt	Limited to 16 characters.
#3	Encrypted password	Does not include salt, unlike traditional Unix crypt( ) function.

4.3.2.4 The shadow password and master password files

Although changes to the encrypted password system (as described in the previous section) have improved the security of encrypted passwords, they have failed to fundamentally address the weakness exploited by password crackers: people pick passwords that are easy to guess. If an attacker can obtain a copy of the password file, it is a simple matter to guess passwords, perform the encryption transform, and compare against the file.

Ultimately, the best way to deal with the problem of poorly-chosen passwords is to eliminate reusable passwords entirely by using one-time passwords, some form of biometrics, or a token-based authentication system. Because such systems can be awkward or expensive, modern Unix systems have adopted a second approach called shadow password files or master password files.

As the name implies, a shadow password file is a secondary password file that shadows the primary password file. On Solaris and Linux systems, the shadow password is usually stored in the file /etc/shadow and contains the encrypted password and a password expiration date. The /etc/shadow file is protected so that it can be read only by the superuser. Thus, an attacker cannot obtain a copy to use in verifying guesses of passwords.

Instead of a shadow password file, FreeBSD uses a master password file. This file, /etc/master.passwd, is a complete password file that includes usernames, passwords, and other account information. The /etc/passwd file is identical to the /etc/master.passwd file, except that all encrypted passwords have been changed to the letter "x".

Mac OS X stores all account information in the NetInfo network-based account management system. Mac OS X does this for all computers, even for standalone computers that are never placed on a network. The version of NetInfo that is supplied in Mac OS 10.0 and 10.1 does not provide for shadow passwords, although the /etc/master.passwd file is present and is used during boot-up.

4.3.3 One-Time Passwords

The most effective way to minimize the danger of bad passwords is not to use conventional passwords at all. Instead, your site can install software and/or hardware to allow one-time passwords. A one-time password is exactly that—a password that is used only once.

There are two popular techniques for implementing one-time passwords:

Hardware tokens: An example is the RSA SecureID card, which displays a new PIN or password for each login. Some token-based systems display a different code every minute. Other token-based systems look like little calculators. When you attempt to log in you are presented with a challenge. You type this challenge into your calculator, type in your personal identification number, and then type the resulting number that is displayed into the computer.
Codebooks: These list valid passwords. Each password is crossed off the list after it is used. S/Key is a popular codebook system.^[18]

^[18] More correctly, it is a one-time pad and not a codebook.

One-time passwords can be implemented as a replacement for conventional passwords or in addition to them. In a typical S/Key environment, you enter the S/Key password instead of your standard Unix password. For example:

login: darrel
Password: says rusk wag hut gwen loge

Last login: Wed Jul  5 08:11:33 from r2.nitroba.com
You have new mail.
%

All of these one-time password systems provide an astounding improvement in security over the conventional system. Unfortunately, because they require either the installation of special software or the purchase of additional hardware, they are not as widespread at this time in the Unix marketplace as they should be. However, many major companies and government agencies have moved to using these one-time methods. (See Table 19-1 for additional details.)

4.3.4 Public Key Authentication

Another approach to solving the problem of passwords is to do away with them entirely and use an alternative authentication system. One popular authentication system that has been used is recent years is based on public key cryptography (described in Chapter 7 ).

In a public key authentication system, each user generates a pair of "keys"—two long numbers with the interesting property that a message encoded with one of the keys can be decoded only using the other key. The user keeps one of the keys private on his local computer (and often protects its privacy by encrypting the key itself with a password), and provides the other, public key to the remote server. When the user wants to log into the server, the server selects a random number, encodes it with the user's public key, and sends it to the user. By decrypting the random number using his private key and returning it to the server (possibly re-encrypted with the server's public key), the user proves that he is in possession of the private key and is therefore authentic. In a similar fashion, the server can authenticate itself to the user, so that the user is sure that he's logging into the correct machine.

Public key authentication systems have two fundamental problems. The first problem is the management of private keys. Private keys must be kept secure at all costs. Typically, private keys are encrypted with a passphrase to protect them, but all of the caveats about choosing a good password (and not transmitting it where others can eavesdrop) apply.

The second problem is the certification of public keys. If an attacker can substitute his public key for someone else's (or for that of a server to which you wish to connect) all your communication will be visible to the attacker. One solution to this problem is to use a secure channel to exchange public keys. With the Secure Shell (ssh), the public key is merely copied to the remote system (after logging in with a password or another non-public key method) and put into a file in the user's home directory called ~/.ssh/authorized_keys.

A more sophisticated technique for distributing public keys involves the creation of a public key infrastructure (PKI). A group of users and system administrators could all certify their keys to one another in person, or each could have his own key certified by a common person or organization that everyone trusts to verify the identities associated with the keys. SSL, the Secure Socket Layer, provides transparent support for PKI.

The Care and Feeding of Passwords

2008-06-02T05:20:00.000-07:00

Although passwords are an important element of computer security, users often receive only cursory instructions about selecting them.

If you are a user, be aware that by picking a bad password—or by revealing your password to an untrustworthy individual—you are potentially compromising your entire computer's security. If you are a system administrator, you should make sure that all of your users are familiar with the issues raised in this section.

4.2.1 Bad Passwords: Open Doors

A bad password is any password that is easily guessed.

Bad Passwords

When picking passwords, avoid the following:

Your name, spouse's name, or partner's name
Your pet's name or your child's name
Names of close friends or coworkers
The name of your company, school, department, or group
Names of your favorite fantasy characters
Your boss's name
Anybody's name
The name of the operating system you're using
Information in the GECOS field of your passwd file entry (discussed later in this chapter)
The hostname of your computer
Your phone number or your license plate number
Any part of your Social Security number
Anybody's birth date
Other information easily obtained about you (e.g., address, alma mater)
Words such as wizard, guru, gandalf, and so on
Any username on the computer in any form (as is, capitalized, doubled, etc.)
A word in the English dictionary or in a foreign dictionary
Place names or any proper nouns
Passwords of all the same letter
Simple patterns of letters on the keyboard, like qwerty
Any of the above spelled backwards
Any of the above followed or prepended by a single digit

In the movie Real Genius, a computer recluse named Laszlo Hollyfeld breaks into a top-secret military computer over the telephone by guessing passwords. Laszlo starts by typing the password AAAAAA, then trying AAAAAB, then AAAAAC, and so on, until he finally finds the password that matches.

Real-life computer crackers are far more sophisticated. Instead of typing each password by hand, attackers use their computers to open network connections (or make phone calls) then try the passwords, automatically retrying when they are disconnected. Instead of trying every combination of letters, starting with AAAAAAhit lists of common passwords such as wizard or demo. Even a modest home computer with a good password-guessing program can try many thousands of passwords in less than a day's time. Some hit lists used by crackers are several hundred thousand words in length, and include words in many different languages.^[6] Therefore, a password that anybody on the planet^[7] (or whatever), attackers use might use for a password is probably a bad password choice for you.

^[6] In contrast, if you were to program a home computer to try all 6-letter combinations from AAAAAA to ZZZZZZ, it would have to try 308,915,776 different passwords. Guessing one password per second, that would require nearly 10 years. Many Unix systems make this process even slower by introducing delays between login attempts.

^[7] If you believe that beings from other planets have access to your computer account, then you should not pick a password that they can guess, either, although this may be the least of your problems.

What's a popular and bad password? Some examples are your name, your partner's name, or your parents' names. Other bad passwords are these names backwards or followed by a single digit. Short passwords are also bad, because there are fewer of them: they are, therefore, more easily guessed. Especially bad are "magic words" from computer games, such as xyzzy. Magic words look secret and unguessable, but in fact they are widely known. Other bad choices include phone numbers, characters from your favorite movies or books, local landmark names, favorite drinks, or famous computer scientists (see the sidebar Bad Passwords for still more bad choices). These words backwards or capitalized are also weak. Replacing the letter "l" (lowercase "L") with "1" (numeral one), the letter "o" with "0" (numeral zero), or "E" with "3," adding a digit to either end, or other simple modifications of common words are also weak. Words in other languages are no better. Dictionaries for dozens of languages are available for download on the Internet, including Klingon! There are also dictionaries available that consist solely of words frequently chosen as passwords.

Many versions of Unix make a minimal attempt to prevent users from picking bad passwords. For example, under some versions of Unix, if you attempt to pick a password with fewer than six letters or letters that are all the same case, the passwd program will ask the user to "Please pick a different password" followed by some explanation of the local requirements for a password. After three tries, however, some versions of the passwd program relent and let the user pick a short one. Better versions allow the administrator to require a minimum number of letters, a requirement for nonalphabetic characters, and other restrictions. However, some administrators turn these requirements off because users complain about them! Users will likely complain more loudly if their computers are broken into.

4.2.2 Smoking Joes

Surprisingly, a significant percentage of all computers that do not explicitly check for bad passwords contain at least one account in which the username and the password are the same or extremely similar. Such accounts are often called "Joes." Joe accounts are easy for crackers to find and trivial to penetrate. Attackers can find an entry point into far too many systems simply by checking every account to see whether it is a Joe account. This is one reason why it is dangerous for your computer to make a list of all of the valid usernames available to the outside world.

4.2.3 Good Passwords: Locked Doors

Good passwords are passwords that are difficult to guess. The best passwords are difficult to guess because they include some subset of the following characteristics:

Have both uppercase and lowercase letters
Have digits and/or punctuation characters as well as letters
May include some control characters and/or spaces^[8]

^[8] In some cases, using spaces may be problematic. An attacker who is in a position to listen carefully can distinguish the sound of the space bar from the sound of other keys. Similarly, Shift or Control key combinations have a distinctive sound, but there are many shifted characters and only one space.
Are easy to remember, so they do not have to be written down
Are seven or eight characters long.
Can be typed quickly, so somebody cannot determine what you type by watching over your shoulder

It's easy to pick a good password. Here are some suggestions:

Take two short words and combine them with a special character or a number, like robot4my or eye-con.
Put together an acronym that's special to you, like Anotfsw (Ack, none of this fancy stuff works), aUpegcbm (All Unix programmers eat green cheese but me), or Ttl*Hiww (Twinkle, twinkle, little star. How I wonder what . . . ).
Create a nonsense word by alternating consonant and vowel sounds, like huroMork. These words are usually easy to pronounce and remember.

Of course, robot4my, eye-con, Anotfsw, Ttl*Hiww, huroMork, and aUpegcbm are now all bad passwords because they've been printed here.

Number of Passwords

If you exclude a few of the Control characters that should not be used in a password, it is still possible to create more than 5,000,000,000,000,000 unique 8-character passwords in standard Unix.

Combining dictionaries from 10 different major languages, plus those words reversed, capitalized, with a trailing digit appended, and otherwise slightly modified results in less than 5,000,000 words. Adding a few thousand names and words from popular culture hardly changes that.

From this, we can see that users who pick weak passwords are making it easy for attackers—they reduce the search space to less than .000000001% of the possible passwords!

One study of passwords chosen in an unconstrained environment revealed that users chose passwords with Control characters only 1.4% of the time, and punctuation and space characters less than 6% of the time. All of the characters !@#$%^&*( )_-+=[ ]|\;:"?/,.< >'~' can be used in passwords too; although, some systems may treat the "\", "#", and "@" symbols as escape (literal), erase, and kill, respectively. (See the footnote to the earlier sidebar entitled "Password: ChangeMe" for a list of the control characters that should not be included in a password.)

Next time one of your users complains because of the password selection restrictions you have in place and proclaims, "I can't think of any password that isn't rejected by the program!", you might want to show him this page.

4.2.4 Password Synchronization: Using the Same Password on Many Machines

If you have several computer accounts, you may wish to have the same password on every machine, so you have less you need to remember. This is called password synchronization.

Password synchronization can increase security if the synchronization allows you to use a good password that is hard to guess. Systems that provide for automated password synchronization make it easy to change your password and have that change reflected everywhere.

On the other hand, password synchronization can decrease security if the password is compromised—suddenly all of your accounts will be vulnerable! Even worse, with password synchronization you may not even know that your password has been compromised!

Password synchronization is also problematic for usernames and passwords that are used for web sites. Many people will use the same username and password at many web sites—even web sites that are potentially being run by untrustworthy individuals or organizations. A simple way to capture usernames and passwords is to set up a web site that offers "a chance of winning $10,000" to anybody who registers with an email address and sets up a password upon entry.

If you are thinking of using the same password on many machines, here are some points to consider:

One common approach used by people with accounts on many machines is to have a base password that can be modified for each different machine. For example, your base password might be kxyzzy followed by the first letter of the name of the computer you're using. On a computer named athena your password would be kxyzzya, while on a computer named ems your password would be kxyzzye. (Don't, of course, use this exact method of varying your passwords.)
Another common approach is to create a different, random password for each machine. Store these passwords in a file that is encrypted—either manually encrypted with a program such as PGP, or automatically encrypted using a "password keeper" program.
To simplify access to remote systems, configure your remote accounts for ssh-based access using your ssh key. Make sure that this key is kept encrypted using an ssh passphrase. For day-to-day use, the ssh passphrase is all that needs to be remembered. However, for special cases or when changing the password, you can refer to your encrypted file of all the passwords. See the manual page for ssh-keygen for specific instructions.

4.2.5 Writing Down Passwords

In the movie War Games, there is the canonical story about a high school student who breaks into his school's academic computer and changes his grades; he does this by walking into the school's office, looking at the academic officer's terminal, and noting that the telephone number, username, and password are written on a Post-It note.

Unfortunately, the fictional story has actually happened—in fact, it has happened hundreds of times over.

Users are admonished to "never write down your password." The reason is simple enough: if you write down your password, somebody else can find it and use it to break into your computer. A password that is memorized is more secure than the same password written down, simply because there is less opportunity for other people to learn it. On the other hand, a password that must be written down to be remembered is quite likely a password that is not going to be guessed easily.^[9] If you write your password on something kept in your wallet, the chances of somebody who steals your wallet using the password to break into your computer account are remote indeed.^[10]

^[9] We should note that in the 12 years since we originally wrote this, we have added lots more accounts and passwords and have more frequent "senior moments." Thus, we perhaps should be a little less emphatic about this point.

^[10] Unless, of course, you happen to be an important person, and your wallet is stolen or rifled as part of an elaborate plot. In their book Cyberpunks, authors John Markoff and Katie Hafner describe a woman named "Susan Thunder" who broke into military computers by doing just that: she would pick up an officer at a bar and go home with him. Later that night, while the officer was sleeping, Thunder would get up, go through the man's wallet, and look for telephone numbers, usernames, and passwords.

If you must write down your password, then at least follow a few precautions:

When you write it down, don't identify your password as being a password.
Don't include the name of the account, network name, or phone number of the computer on the same piece of paper as your password.
Don't attach the password to your terminal, keyboard, or any part of your computer.
Don't write your actual password. Instead, disguise it by mixing in other characters or by scrambling the written version of the password in a way that you can remember. For example, if your password is Iluvfred, you might write fredIluv or vfredxyIu or perhaps Last week, I lost Uncle Vernon's `fried rice &eggplant delight' recipe--remember to call him after 3:00 p.m.—to throw off a potential wallet-snatcher.^[11]

^[11] We hope that last one required some thought. The 3:00 p.m. means to start with the third word and take the first letter of every word. With some thought, you can come up with something equally obscure that you will remember.

Of course, you can always encrypt your passwords in a handy file on a machine where you remember the password. Many people store their passwords in an encrypted form on a PDA (handheld computer). The only drawback to this approach is when you can't get to your file, or your PDA has gone missing (or its batteries die)—how do you log on to report the problem?

Here are some other things to avoid:

Don't record a password online (in a file, database, or email message), unless the password is encrypted.
Likewise, never send a password to another user via electronic mail. In The Cuckoo's Egg, Cliff Stoll tells of how a single intruder broke into system after system by searching for the word password in text files and electronic mail messages. With this simple trick, the intruder learned of the passwords of many accounts on many different computers across the country.
Don't use your login password as the password of application programs. For instance, don't use your login password as your password to an online MUD (multiuser dungeon) game or for a web server account. The passwords in those applications are controlled by others and may be visible to the wrong people.
Don't use the same password for different computers managed by different organizations. If you do, and an attacker learns the password for one of your accounts, all will be compromised.

This last "don't" is very difficult to follow in practice.

Logging in with Usernames and Passwords

2008-06-02T05:18:00.000-07:00

Every person who uses a Unix computer should have her own account. An account is identified by a user ID number (UID) that is associated with one or more usernames (also known as account names). Traditionally, each account also has a secret password associated with it to prevent unauthorized use. You need to know both your username and your password to log into a Unix system.

4.1.1 Unix Usernames

The username is an identifier: it tells the computer who you are. In contrast, a password is an authenticator: you use it to prove to the operating system that you are who you claim to be. A single person can have more than one Unix account on the same computer. In this case, each account would have its own username.

Standard Unix usernames may be between one and eight characters long, although many Unix systems today allow usernames that are longer. Within a single Unix computer, usernames must be unique: no two users can have the same one. (If two people did have the same username on a single system, then they would really be sharing the same account.) Traditionally, Unix passwords were also between one and eight characters long, although most Unix systems now allow longer passwords as well. Longer passwords are generally more secure because they are harder to guess. More than one user can theoretically have the same password, although if they do, that usually indicates that both users have picked a bad password.

A username can be any sequence of characters you want (with some exceptions), and does not necessarily correspond to a real person's name.

Some versions of Unix have problems with usernames that do not start with a lowercase letter or that contain special characters such as punctuation or control characters. Usernames containing certain unusual characters will also cause problems for various application programs, including some network mail programs. For this reason, many sites allow only usernames that contain lowercase letters and numbers and further require that all usernames start with a letter.

Your username identifies you to Unix in the same way that your first name identifies you to your friends. When you log into the Unix system, you tell it your username in the same way that you might say, "Hello, this is Sabrina," when you pick up the telephone.^[2] Most systems use the same identifier for both usernames and email addresses. For this reason, organizations that have more than one computer often require people to use the same username on every machine to minimize confusion.

^[2] Even if you aren't Sabrina, saying that you are Sabrina identifies you as Sabrina. Of course, if you are not Sabrina, your voice will probably not authenticate you as Sabrina, provided that the person you are speaking with knows what Sabrina actually sounds like.

There is considerable flexibility in choosing a username. For example, John Q. Random might have any of the following usernames; they are all potentially valid:

john

johnqr

johnr

jqr

jqrandom

jrandom

random

randomjq

Alternatively, John might have a username that appears totally unrelated to his real name, like avocado or t42. Having a username similar to your own name is merely a matter of convenience.

In some cases, having an unrelated name may be a desired feature because it either masks your identity in email and online chat rooms, or projects an image different from your usual one: tall62, fungirl, anonymus, svelte19, and richguy. Of course, as we noted in the last chapter, "handles" that don't match one's real name can also be used to hide the true identity of someone doing something unethical or illegal. Be cautious about drawing conclusions about someone based on the email name or account name that they present.

Most organizations require that usernames be at least three characters long. Single-character usernames are simply too confusing for most people to deal with, no matter how easy you might think it would be to be user i or x. Usernames that are two characters long are also confusing for some people, because they usually don't provide enough information to match a name in memory: who was zt@ex.com, anyway? In general, names with little intrinsic meaning, such as t42xp96wl, can also cause confusion because they are more difficult for correspondents to remember.

Some organizations assign usernames using standardized rules, such as the first initial of a person's first name and then the first six letters of their last name, optionally followed by a number. Other organizations let users pick their own names. Some organizations and online services assign an apparently random string of characters as the usernames; although this is generally not popular, it can improve security—especially if these usernames are not used for electronic mail. Although some randomly generated strings can be hard to remember, there are several algorithms that generate easy-to-remember random strings by using a small number of mnemonic rules; typical usernames generated by these systems are xxp44 and acactt. If you design a system that gives users randomly generated usernames, it is a good idea to let people reject a username and ask for another, lest somebody gets stuck with a hard-to-remember username like xp9uu6wi.

Unix also has special accounts that are used for administrative purposes and special system functions. These accounts are not normally used by individual users.

4.1.2 Authenticating Users

After you tell Unix who you are, you must prove your identity to a certain degree of confidence (trust). This process is called authentication. Classically, there are three different ways that you can authenticate yourself to a computer system, and you use one or more of them each time:

You can tell the computer something that you know (for example, a password).
You can present the computer with something you have (for example, a card key).
You can let the computer measure something about you (for example, your fingerprint).

None of these systems is foolproof. For example, by eavesdropping on your terminal line, somebody can learn your password. By attacking you at gunpoint, somebody can steal your card key. And if your attacker has a knife, you might even lose your finger! In general, the more trustworthy the form of authentication, the more aggressive an attacker must be to compromise it. In the past, the most trustworthy authentication techniques have also been the most difficult to use, although this is slowly changing.

4.1.3 Authenticating with Passwords

Passwords are the simplest form of authentication: they are a secret that you share with the computer. When you log in, you type your password to prove to the computer that you are who you claim to be. The computer ensures that the password you type matches the account that you have specified. If it matches, you are allowed to proceed.

Unix does not display your password as you type it. This gives you extra protection if the transcript of your session is being logged or if somebody is watching over your shoulder as you type—a technique that is sometimes referred to as shoulder surfing.

Why Authenticate?

Traditionally desktop personal computers running the Windows or Macintosh operating systems, handheld computers, and personal organizers did not require that users authenticate themselves before the computer provided the requested information. The fact that these computers employed no passwords or other authentication techniques made them easier to use.

Likewise, many of the research groups that originally developed the Unix operating system did not have passwords for individual users—often for the same reason that they shied away from locks on desks and office doors. In these environments, trust, respect, and social convention were very powerful deterrents to information theft and destruction. When computer systems required passwords, often times many people shared the same password—password, for example.

Unfortunately, the lack of authentication made these computers easier for many people to use—this included both the machine's primary user and anybody else who happened to be in the area. As these systems were connected to modems or external networks, the poor authentication practices that had grown up in the closed environment became a point of vulnerability, especially when other systems based their trust on the authenticity of the identity determined locally. Vulnerabilities frequently led to successful attacks. There have been many cases in which a single easily compromised account has endangered the security of an entire installation or network.

In today's highly networked world, proper authentication of authorized users is a core requirement of any computer that is trusted with confidential information. The challenge that computer developers now face is to produce systems that provide strong authentication while simultaneously providing ease of use.

Conventional passwords have been part of Unix since its early years. The advantage of this system is that it runs without any special equipment, such as smartcard readers or fingerprint scanners.

The disadvantage of conventional passwords is that they are easily captured and reused—especially in a network-based environment. Although passwords can be used securely and effectively, doing so requires constant vigilance to make sure that an unencrypted password is not inadvertently sent over the network, allowing it to be captured with a password sniffer. Passwords can also be stolen if they are typed on a computer that has been compromised with a keystroke recorder. Today, even unsophisticated attackers can use such tools to capture passwords. Indeed, the only way to safely use a Unix computer remotely over a network such as the Internet is to use one-time passwords, encryption, or both (see Section 4.3.3 later in this chapter and also see Chapter 7).^[3]

^[3] Well-chosen passwords are still quite effective for most standalone systems with hardwired terminals, and when used in cryptographic protocols with mechanisms to prevent replay attacks.

Unfortunately, we live in an imperfect world, and most Unix systems continue to depend upon reusable passwords for user authentication. Be careful!

4.1.3.1 Entering your password

When you log in, you tell the computer who you are by typing your username at the login prompt (the identification step). You then type your password (in response to the password prompt) to authenticate that you are who you claim to be. For example:

login: rachel
password: luV2-fred

Unix does not display your password when you type it.

If the password that you supply with your username corresponds to the password that is on file for the provided username, Unix logs you in and gives you full access to the user's files, commands, and devices. If the username and the password do not match, Unix does not log you in.

On some versions of Unix, if somebody tries to log into an account and supplies an invalid password several times in succession, that account will become locked. A locked account can be unlocked only by the system administrator. Locking has three functions:

It protects the system from attackers who persist in trying to guess a password; before they can guess the correct password, the account is shut down.
It lets you know that someone has been trying to break into your account.
It lets your system administrator know that someone has been trying to break into the computer.

If you find yourself locked out of your account, you should contact your system administrator and get your password changed to something new. Don't change your password back to what it was before you were locked out.

The automatic lockout feature can prevent unauthorized use, but it can also be used to conduct denial of service attacks, or by an attacker to lock selected users out of the system so as to prevent discovery of his actions. A practical joker can use it to annoy fellow employees or students. And you can accidentally lock yourself out if you try to log in too many times before you've had your morning coffee.

In our experience, the disadvantages of indefinite automatic lockouts outweigh the benefits. A much better method is to employ an increasing delay mechanism in the login. After a fixed number of unsuccessful logins, an increasing delay can be inserted between each successive prompt. Implementing such delays in a network environment requires maintaining a record of failed login attempts, so that the delay cannot be circumvented by an attacker who merely disconnects from the target machine and reconnects.

4.1.3.2 Changing your password

You can change your password with the Unix passwd command. You will first be asked to type your old password, then a new one. By asking you to type your old password first, passwd prevents somebody from walking up to a terminal that you left yourself logged into and then changing your password without your knowledge.

Unix makes you type the new password twice:

% passwd
Changing password for sarah.
Old password:tuna4fis

New password: nosSMi32

Retype new password: nosSMi32

%

If the two passwords you type don't match, your password remains unchanged. This is a safety precaution: if you made a mistake typing the new password and Unix only asked you once, then your password could be changed to some new value and you would have no way of knowing that value.

On systems that use Sun Microsystems NIS or NIS+, you may need to use the command yppasswd or nispasswd to change your password. Except for having different names, these programs work in the same way as passwd. However, when they run, they update your password in the network database with NIS or NIS+. When this happens, your password will be immediately available on other clients on the network. With NIS, your password will be distributed during the next regular update.

The -r option to the passwd command can also be used under Solaris. To change NIS or NIS+ passwords, the format would be passwd -r nis or passwd -r nisplus, respectively. It is possible to have a local machine password that is different from the one in the network database, and that would be changed with passwd -r files.

Even though passwords are not echoed when they are printed, the Backspace or Delete key (or whatever key you have bound to the "erase" function) will still delete the last character typed, so if you make a mistake, you can correct it.

Once you have changed your password, your old password will no longer work. Do not forget your new password! If you forget your new password, you will need to have the system administrator set it to something you can use to log in and try again.^[4]

^[4] And if you are the system administrator, you'll have to log in as the superuser to change your password. If you've forgotten the superuser password, you may need to take drastic measures to recover.

If your system administrator gives you a new password, immediately change it to something else that only you know! Otherwise, if your system administrator is in the habit of setting the same password for forgetful users, your account may be compromised by someone else who has had a temporary lapse of memory; see Password: ChangeMe for an example.

If you are a system manager and you need to change a user's password, do not change the user's password to something like changeme or password, and then rely on the user to change their password to something else. Many users will not take the time to change their passwords but will, instead, continue to use the password that you have inadvertently "assigned" to them. Give the user a good password, and give that user a different password from every other user whose password you have reset.

4.1.3.3 Verifying your new password

After you have changed your password, try logging into your account with the new password to make sure that you've entered the new password properly. Ideally, you should do this without logging out, so you will have some recourse if you did not change your password properly. This is especially crucial if you are logged in as root and you have just changed the root password!

Password: ChangeMe

At one major university we know about, it was commonplace for students to change their passwords and then be unable to log into their accounts. Most often this happened when students tried to put control characters into their passwords.^[5] Other times, students mistyped the password and were unable to retype it again later. More than a few got so carried away making up a fancy password that they couldn't remember their passwords later.

Well, once a Unix password is entered, there is no way to decrypt it and recover it. The only recourse is to have someone change the password to another known value. Thus, the students would bring a picture ID to the computing center office, where a staff member would change the password to ChangeMe and instruct them to immediately go down the hall to a terminal room to do exactly that.

Late one semester shortly after the Internet worm incident (which occurred in November of 1988), one of the staff decided to try running a password cracker (see Chapter 19) to see how many student account passwords were weak. Much to the surprise of the staff member, dozens of the student accounts had a password of ChangeMe. Furthermore, at least one of the other staff members also had that as a password! The policy soon changed to one in which forgetful students were forced to enter a new password on the spot.

Some versions of the passwd command support a special -f flag. If this flag is provided when the superuser changes a person's password, that user is forced to change his or her password the very next time he logs into the system. It's a good option for system administrators to remember.

^[5] The control characters ^@, ^C, ^G, ^H, ^J, ^M, ^Q, ^S, and ^[ should not be put in passwords, because they can be interpreted by the system. If your users will log in using xdm, users should avoid all control characters, as xdm often filters them out. You should also beware of control characters that may interact with your terminal programs, terminal concentrator monitors, and other intermediate systems you may use; for instance, the ~ character is often used as an escape character in ssh and rsh sessions. Finally, you may wish to avoid the # and @ characters, as some Unix systems still interpret these characters with their ancient use as erase and kill characters.

One way to try out your new password is to use the su command. Normally, the su command is used to switch to another account. But as the command requires that you type the password of the account to which you are switching, you can effectively use the su command to test the password of your own account.

% /bin/su nosmis
password: mypassword

%

(Of course, instead of typing nosmis and mypassword , use your own account name and password.)

If you're using a machine that is on a network, you can use the telnet, rlogin, or ssh programs to loop back through the network to log in a second time by typing:

% ssh -l dawn localhost

dawn@loaclhost's password: w3kfsc!

Last login: Sun Feb 3 11:48:45 on ttyb
%

You can replace localhost in the above example with the name of your computer. This method is also useful when testing a change in the root password, as the su command does not prompt for a password when run by root.

If you try one of the earlier methods and discover that your password is not what you thought it was, you have a definite problem. To change the password to something you do know, you will need the current password. However, you don't know that password! You will need the help of the system administrator to fix the situation. (That's why you shouldn't log out—if the time is 2:00 a.m. on Saturday, you might not be able to reach the administrator until Monday morning, and you might want to get some work done before then.)

The superuser (user root) can't decode the password of any user. However, the system administrator can help you when you don't know what you've set your password to by using the superuser account to set your password to something known.

If you get email from your system manager advising you that there are system problems and that you should immediately change your password to tunafish (or some other value), disregard the message and report it to your system management. These kinds of email messages are frequently sent by computer criminals to novice users. The hope is that the novice user will comply with the request and change his password to the one that is suggested—often with devastating results.

4.1.3.4 Changing another user's password

If you are running as the superuser (or the network administrator, in the case of NIS+), you can set the password of any user, including yourself, without supplying the old password. You do this by supplying the username to the passwd command when you invoke it:

# passwd cindy
New password: NewR-pas

Retype new password: NewR-pas

#

Risk Assessment

2008-06-02T05:17:00.000-07:00

The first step in improving the security of your system is to answer these basic questions:

What am I trying to protect and how much is it worth to me?
What do I need to protect against?
How much time, effort, and money am I willing to expend to obtain adequate protection?

These questions form the basis of the process known as risk assessment. Risk assessment is a very important part of the computer security process. You cannot formulate protections if you do not know what you are protecting and what you are protecting those things against! After you know your risks, you can then plan the policies and techniques that you need to implement to reduce those risks.

For example, if there is a risk of a power failure and if availability of your equipment is important to you, you can reduce this risk by installing an uninterruptable power supply (UPS).

3.2.1 Steps in Risk Assessment

Risk assessment involves three key steps:

Identifying assets and their value
Identifying threats
Calculating risks

There are many ways to go about this process. One method with which we have had great success is a series of in-house workshops. Invite a broad cross-section of knowledgeable users, managers, and executives from throughout your organization. Over the course of a series of meetings, compose your lists of assets and threats. Not only does this process help to build a more complete set of lists, it also helps to increase awareness of security in everyone who attends.

An actuarial approach is more complex than necessary for protecting a home computer system or very small company. Likewise, the procedures that we present here are insufficient for a large company, a government agency, or a major university. In cases such as these, many companies turn to outside consulting firms with expertise in risk assessment, some of which use specialized software to do assessments.

3.2.1.1 Identifying assets

Draw up a list of items you need to protect. This list should be based on your business plan and common sense. The process may require knowledge of applicable law, a complete understanding of your facilities, and knowledge of your insurance coverage.

Items to protect include tangibles (disk drives, monitors, network cables, backup media, manuals, etc.) and intangibles (ability to continue processing, your customer list, public image, reputation in your industry, access to your computer, your system's root password, etc.). The list should include everything that you consider to be of value. To determine if something is valuable, consider what the loss or damage of the item might cost in terms of lost revenue, lost time, or the cost of repair or replacement.

Some of the items that should probably be in your asset list include:

Tangibles

Computers
Proprietary data
Backups and archives
Manuals, guides, books
Printouts
Commercial software distribution media
Communications equipment and wiring
Personnel records
Audit records

Intangibles

Safety and health of personnel
Privacy of users
Personnel passwords
Public image and reputation
Customer/client goodwill
Processing availability
Configuration information

You should take a larger view of these and related items rather than simply considering the computer aspects. If you are concerned about someone reading your internal financial reports, you should be concerned regardless of whether they read them from a discarded printout or snoop on your email.

3.2.1.2 Identifying threats

The next step is to determine a list of threats to your assets. Some of these threats will be environmental, and include fire, earthquake, explosion, and flood. They should also include very rare but possible events such as structural failure in your building, or the discovery of asbestos in your computer room that requires you to vacate the building for a prolonged time. Other threats come from personnel and from outsiders. We list some examples here:

Illness of key people
Simultaneous illness of many personnel (e.g., flu epidemic)
Loss (resignation/termination/death) of key personnel
Loss of phone/network services
Loss of utilities (phone, water, electricity) for a short time
Loss of utilities (phone, water, electricity) for a prolonged time
Lightning strike
Flood
Theft of disks or tapes
Theft of key person's laptop computer
Theft of key person's home computer
Introduction of a virus
Bankruptcy of a key vendor or service provider
Hardware failure
Bugs in software
Subverted employees
Subverted third-party personnel (e.g., vendor maintenance)
Labor unrest
Political terrorism
Random "hackers" getting into your machines
Users posting inflammatory or proprietary information on the Web

3.2.2 Review Your Risks

Risk assessment should not be done only once and then forgotten. Instead, you should update your assessment periodically. In addition, the threat assessment portion should be redone whenever you have a significant change in operation or structure. Thus, if you reorganize, move to a new building, switch vendors, or undergo other major changes, you should reassess the threats and potential losses.

Planning Your Security Needs

2008-06-02T05:16:00.000-07:00

There are many different kinds of computer security, and many different definitions. Rather than present a formal definition, this book takes a practical approach and discusses the categories of protection you should consider. Basically, we a computer is secure if it behaves the way you expect it to. We believe that secure computers are usable computers and, likewise, that computers that cannot be used, for whatever the reason, are not very secure.

3.1.1 Types of Security

Within our broad definition of computer security, there are many different types of security that both users and administrators of computer systems need to be concerned about:

Confidentiality: Protecting information from being read or copied by anyone who has not been explicitly authorized by the owner of that information. This type of security includes not only protecting the information in toto, but also protecting individual pieces of information that may seem harmless by themselves but can be used to infer other confidential information.
Data integrity: Protecting information (including programs) from being deleted or altered in any way without the permission of the owner of that information. Information to be protected also includes items such as accounting records, backup tapes, file creation times, and documentation.
Availability: Protecting your services so they're not degraded or made unavailable (crashed) without authorization. If the systems or data are unavailable when an authorized user needs them, the result can be as bad as having the information that resides on the system deleted.
Consistency: Making sure that the system behaves as expected by the authorized users. If software or hardware suddenly starts behaving radically different from the way it used to behave, especially after an upgrade or a bug fix, a disaster could occur. Imagine if your ls command occasionally deleted files instead of listing them! This type of security can also be considered as ensuring the correctness of the data and software you use.
Control: Regulating access to your system. If unknown and unauthorized individuals (or software) are found on your system, they can create a big problem. You must worry about how they got in, what they might have done, and who or what else has also accessed your system. Recovering from such episodes can require considerable time and expense in rebuilding and reinstalling your system, and verifying that nothing important has been changed or disclosed—even if nothing actually happened.
Audit: As well as worrying about unauthorized users, you need to realize that authorized users sometimes make mistakes, or even commit malicious acts. In such cases, you need to determine what was done, by whom, and what was affected. The only sure way to achieve these results is by having some incorruptible record of activity on your system that positively identifies the actors and actions involved. In some critical applications, the audit trail may be extensive enough to allow "undo" operations to help restore the system to a correct state.

Although all of these aspects of security are important, different organizations will view each with a different amount of importance. This variance is because different organizations have different security concerns, and must set their priorities and policies accordingly. For example:

A banking environment: In such an environment, integrity, control, and auditability are usually the most critical concerns, while confidentiality and availability are less important.
A national defense-related system that processes classified information: In such an environment, confidentiality may come first, and availability last. In some highly classified environments, officials may prefer to blow up a building rather than allow an attacker to access the information contained within that building's walls.
A university: In such an environment, integrity and availability may be the most important requirements. It is more important to ensure that students can work on their papers, than that administrators can track the precise times their students accessed their accounts.

If you are a security administrator, you need to thoroughly understand the needs of your operational environment and users. You then need to define your procedures accordingly. Not everything we describe in this book will be appropriate in every environment.

3.1.2 Trust

Security professionals generally don't refer to a computer system as being "secure" or "unsecure."^[1] Instead, we use the word trust to describe our level of confidence that a computer system will behave as expected. This acknowledges that absolute security can never be present. We can only try to approach it by developing enough trust in the overall configuration to warrant using it for the applications we have in mind.

^[1] We use the term unsecure to mean having weak security, and insecure to describe the state of mind of people running unsecure systems.

Developing adequate trust in your computer systems requires careful thought and planning. Operational decisions should be based on sound policy and risk analysis. In the remainder of this chapter, we'll discuss the general procedures for creating workable security plans and policies. The topic is too big, however, for us to provide an in-depth treatment:

If you are at a company, university, or government agency, we suggest that you contact your internal audit and/or risk management department for additional help (they may already have some plans and policies in place that you should know about). You can also learn more about this topic by consulting some of the works referenced in Appendix C. You may also wish to enlist a consulting firm. For example, many large accounting and audit firms now have teams of professionals that can evaluate the security of computer installations.
If you are with a smaller institution or are dealing with a personal machine, you may decide that we cover these issues in greater detail than you actually need. Nevertheless, the information contained in this chapter should help guide you in setting your priorities.

Security and Unix

2008-06-02T05:13:00.000-07:00

Many years ago, Dennis Ritchie said this about the security of Unix: "It was not designed from the start to be secure. It was designed with the necessary characteristics to make security serviceable." In other words, Unix can be secured, but any particular Unix system may not be secure when it is distributed.

Unix is a multiuser, multitasking operating system. Multiuser means that the operating system allows many different people to use the same computer at the same time. Multitasking means that each user can run many different programs simultaneously.

One of the natural functions of such operating systems is to prevent different people (or programs) using the same computer from interfering with each other. Without such protection, a wayward program (perhaps written by a student in an introductory computer science course) could affect other programs or other users, could accidentally delete files, or could even crash (halt) the entire computer system. To keep such disasters from happening, some form of computer security has always had a place in the Unix design philosophy.

But Unix security provides more than mere memory protection. Unix has a sophisticated security system that controls the ways users access files, modify system databases, and use system resources. Unfortunately, those mechanisms don't help much when the systems are misconfigured, are used carelessly, or contain buggy software. Nearly all of the security holes that have been found in Unix over the years have resulted from these kinds of problems rather than from shortcomings in the intrinsic design of the system. Thus, nearly all Unix vendors believe that they can (and perhaps do) provide a reasonably secure Unix operating system. We believe that Unix systems can be fundamentally more secure than other common operating systems. However, there are influences that work against better security in the Unix environment.

2.2.1 Expectations

The biggest problem with improving Unix security is arguably one of expectations. Many users have grown to expect Unix to be configured in a particular way. Their experience with Unix in academic, hobbyist, and research settings has always been that they have access to most of the directories on the system and that they have access to most commands. Users are accustomed to making their files world-readable by default. Users are also often accustomed to being able to build and install their own software, frequently requiring system privileges to do so. The trend in "free" versions of Unix for personal computer systems has amplified these expectations.

Unfortunately, all of these expectations are contrary to good security practice in the business world. To have stronger security, system administrators must often curtail access to files and commands that are not required for users to do their jobs. Thus, someone who needs email and a text processor for his work should not also expect to be able to run the network diagnostic programs and the C compiler. Likewise, to heighten security, users should not be able to install software that has not been examined and approved by a trained and authorized individual.

The tradition of open access is strong, and is one of the reasons that Unix has been attractive to so many people. Some users argue that to restrict these kinds of access would make the systems something other than Unix. Although these arguments may be valid, restrictive measures are needed in instances where strong security is required.

At the same time, administrators can strengthen security by applying some general security principles, in moderation. For instance, rather than removing all compilers and libraries from each machine, these tools can be protected so that only users in a certain user group can access them. Users with a need for such access, and who can be trusted to take due care, can be added to this group. Similar methods can be used with other classes of tools, too, such as network monitoring software.

The most critical aspect of enhancing Unix security is to get users themselves to participate in the alteration of their expectations. The best way to meet this goal is not by decree, but through education and motivation. Technical security measures are crucial, but experience has proven repeatedly that people problems are not amenable to technological solutions.

Many users started using Unix in an environment that was less threatening than the one they face today. By educating users about the dangers of lax security, and how their cooperation can help to thwart those dangers, the security of the system is increased. By properly motivating users to participate in good security practice, you make them part of the security mechanism. Better education and motivation work well only when applied together, however; education without motivation may mean that security measures are not actually applied, and motivation without education leaves gaping holes in what is done.

2.2.2 Software Quality

Large portions of the Unix operating system and utilities that people take for granted were written as student projects, or as quick "hacks" by software developers inside research labs or by home hobbyists experimenting with Linux. These programs were not formally designed and tested: they were put together and debugged on the fly.^[7] The result is a large collection of tools and OS code that usually works, but sometimes fails in unexpected and spectacular ways. Utilities were not the only things written by non-experts. Much of BSD Unix, including the networking code, was written by students as research projects of one sort or another—and these efforts sometimes ignored existing standards and conventions. Many of the drivers and extensions to Linux have also been written and tested under varying levels of rigor, and often by programmers with less training and experience than Berkeley graduate students.

^[7] As one of this book's technical reviewers suggests, developers today may be even less likely to spend time in the careful design of code than in the past. In the days when computers ran slowly and compile time was a scarce and valuable resource, time spent ensuring that the program would behave properly when compiled was a good investment. Today, software compilation is so fast that the temptation to repeatedly compile, test, debug, and recompile may lead to a greater reliance on discovering bugs in testing, rather than preventing them in design.

This analysis is not intended to cast aspersions on the abilities of those who wrote all this code; we wish only to point out that most of today's versions of Unix were not created as carefully designed and tested systems. Indeed, a considerable amount of the development of Unix and its utilities occurred at a time when good software engineering tools and techniques were not yet developed or readily available.^[8] The fact that occasional bugs are discovered that result in compromises of the security of some systems should be no surprise! (However, we do note that there is a very large range between, for example, the frequency of security flaws announced for OpenBSD and Red Hat Linux.)

^[8] Some would argue that they are still not available. Few academic environments currently have access to modern software engineering tools because of their cost, and few vendors are willing to provide copies at prices that academic institutions can afford. It is certainly the case that typical home contributors to a *BSD or Linux system code base do not have access to advanced software engineering tools (even if they know how to use them).

Unfortunately, two things are not occurring as a result of the discovery of faults in the existing code. The first is that software designers do not seem to be learning from past mistakes. Consider that buffer overruns (mostly resulting from fixed-length buffers and functions that do not check their arguments) have been recognized as a major problem area for over four decades, yet critical software containing such bugs continues to be written—and exposed. For instance, a fixed-length buffer overrun in the gets( ) library call was one of the major propagation modes of the Internet worm of 1988, yet, as we were working on the second edition of this book in late 1995, news of yet another buffer overrun security flaw surfaced—this time in the BSD-derived syslog( ) library call. During preparation of the third edition in 2002, a series of security advisories were being issued for the Apache web server, the ssh secure login server, and various Microsoft programs, all because of buffer overflows. It is inexcusable that software continues to be formally released with these kinds of problems in place.

A more serious problem than any particular flaw is the fact that few, if any, vendors are performing an organized, well-designed program of design and testing on the software they provide. Although many vendors test their software for compliance with industry "standards," few apparently test their software to see what it does when presented with unexpected data or conditions. According to one study, as much as 40% of the utilities on some machines may have significant problems.^[9] One might think that vendors would be eager to test their new versions of the software to correct lurking bugs. However, as more than one vendor's software engineer has told us, "The customers want their Unix—including the flaws—exactly like every other implementation. Furthermore, it's not good business: customers will pay extra for performance, but not for better testing."

^[9] See the reference to the papers by Barton Miller, et al., given in Appendix C. Note that they found similar problems in Windows, so the problems are clearly not limited to Unix-like systems.

As long as users demand strict conformance of behavior to existing versions of the programs, and as long as software quality is not made a fundamental acquisition criterion by those same users, vendors and producers will most likely do very little to systematically test and fix their software. Formal standards, such as the ANSI C standard and POSIX standard help perpetuate and formalize these weaknesses, too. For instance, the ANSI C standard^[10] perpetuates the gets( ) library call, forcing Unix vendors to support the call, or to issue systems at a competitive disadvantage because they are not in compliance with the standard.

^[10] ANSI X3J11.

We should note that these problems are not confined to the commercial versions of Unix. Many of the open software versions of Unix also incorporate shoddy software. In part, this is because contributors have variable levels of skill and training. Furthermore, these contributors are generally more interested in providing new functionality than they are in testing and fixing flaws in existing code. There are some exceptions, such as the careful code review conducted on OpenBSD, but, paradoxically, the code that is more carefully tested and developed in the open software community also seems to be the code that is least used.

2.2.3 Add-on Functionality Breeds Problems

One final influence on Unix security involves the way that new functionality has been added over the years. Unix is often cited for its flexibility and reuse characteristics; therefore, new functions are constantly built on top of Unix platforms and are eventually integrated into released versions. Unfortunately, the addition of new features is often done without understanding the assumptions that were made with the underlying mechanisms and without concern for the added complexity presented to the system operators and maintainers. Applying the same features and code in a heterogeneous computing environment can also lead to problems.

As a special case, consider how large-scale computer networks such as the Internet have dramatically changed the security ground rules from those under which Unix was developed. Unix was originally developed in an environment where computers did not connect to each other outside of the confines of a small room or research lab. Networks today interconnect hundreds of thousands of machines, and millions of users, on every continent in the world. For this reason, each of us confronts issues of computer security directly: a doctor in a major hospital might never imagine that a postal clerk on the other side of the world could pick the lock on her desk drawer to rummage around her files, yet this sort of thing happens on a regular basis to "virtual desk drawers" on the Internet.

Most colleges and many high schools now grant network access to all of their students as a matter of course. The number of primary schools with network access is also increasing, with initiatives in many U.S. states to put a networked computer in every classroom. Granting telephone network access to a larger number of people increases the chances of telephone abuse and fraud, just as granting widespread computer network access increases the chances that the access will be used for illegitimate purposes. Unfortunately, the alternative of withholding access is equally unappealing. Imagine operating without a telephone because of the risk of receiving prank calls!

The foundations and traditions of Unix network security were profoundly shaped by the earlier, more restricted view of networks, and not by our more recent experiences. For instance, the concept of user IDs and group IDs controlling access to files was developed at a time when the typical Unix machine was in a physically secure environment. On top of this was added remote manipulation commands such as rlogin and rcp that were designed to reuse the user-ID/group-ID paradigm with the concept of "trusted ports" for network connections. Within a local network in a closed lab, using only relatively slow computers, this design (usually) worked well. But now, with the proliferation of workstations and non-Unix machines on international networks, this design, with its implicit assumptions about restricted access to the network, leads to major weaknesses in security.^[11]

^[11] Internet pioneer Bob Metcalf warned of these dangers in 1973, in RFC 602. That warning, and others like it, went largely unheeded.

Not all of these unsecure foundations were laid by Unix developers. The IP protocol suite on which the Internet is based was developed outside of Unix initially, and it was developed without a sufficient concern for authentication and confidentiality. This lack of concern has enabled cases of password sniffing and IP sequence spoofing to occur, and these make news as "sophisticated" attacks.^[12]Chapter 11.) (These attacks are discussed in

^[12] To be fair, the designers of TCP/IP were aware of many of the problems. However, they were more concerned about making everything work so they did not address many of the problems in their design. The problems are really more the fault of people trying to build critical applications on an experimental set of protocols before the protocols were properly refined—a familiar problem.

Another facet of the problem has to do with the "improvements" made by each vendor. Rather than attempting to provide a unified, simple interface to system administration across platforms, each vendor has created a new set of commands and functions. In many cases, improvements to the command set have been available to the administrator. However, there are also now hundreds (perhaps thousands) of new commands, options, shells, permissions, and settings that the administrator of a heterogeneous computing environment must understand and remember. Additionally, many of the commands and options are similar to each other, but have different meanings depending on the environment in which they are used. The result can often be disaster when the poor administrator suffers momentary confusion about the system or has a small lapse in memory. This complexity further complicates the development of tools that are intended to provide cross-platform support and control. For a "standard" operating system, Unix is one of the most nonstandard systems to administer.

That such difficulties arise is both a tribute to Unix and a condemnation. The robust nature of Unix enables it to accept and support new applications by building on the old. However, existing mechanisms are sometimes completely inappropriate for the tasks assigned to them. Rather than being a condemnation of Unix itself, such shortcomings are actually an indictment of the developers for failing to give more consideration to the human and functional ramifications of building on the existing foundation.

Here, then, is a conundrum: to rewrite large portions of Unix and the protocols underlying its environment, or to fundamentally change its structure, would be to attack the very reasons that Unix has become so widely used. Furthermore, such restructuring would be contrary to the spirit of standardization that has been a major factor in the wide acceptance of Unix. At the same time, without re-evaluation and some restructuring, there is serious doubt about the level of trust that can be placed in the system. Ironically, the same spirit of development and change is what has led Unix to its current niche.

2.2.4 The Failed P1003.1e/2c Unix Security Standard

In 1994, work was started within the Unix community on develoing a set of security extensions to the Unix POSIX standard. This standardization effort was known as POSIX P1003.1e/2c.

The ambitious project hoped to create a single Unix security standard comprised of the key security building blocks missing from the underlying Unix design. These included:

Access control lists (ACLs), so that specific individuals or groups of individuals could be given (or denied) access to specific files
Data labeling, allowing classified and confidential data to be labeled as such
Mandatory access control, so that individuals would be unable to override certain security decisions made by the system management
Capabilities that could be used to place restrictions on processes running as the superuser
Standardized auditing and logging

Work on this project continued until October 1997 when, despite good intentions on the part of the participants and the sponsoring vendors, the draft standard was officially withdrawn and the P1003.1e and P1003.2c committees were disbanded. The final drafts of the documents can be downloaded from http://wt.xpilot.org/publications/posix.1e/.

Many factors were responsible for the failure of the P1003.1e/2c standards efforts. Because the standards group sought to create a single standard, areas of disagreement prevented the committee from publishing and adopting smaller standards that represented the areas of consensus. Then a year's worth of work was lost when the "source document" for the standard was lost.

Today, most vendors that sell trusted versions of Unix implement some aspects of the P1003.1e/2c draft standard. Furthermore, the draft has been used as the basis of the Linux capabilities system and the BSD filesystem ACLs. So even though the standards effort was not adopted, it has had a lasting impact.