Abeo Tech Lab

The Entrepreneur’s Handbook – 59 Resources For First Time Entrepreneurs

2009-06-03T16:49:00.000-07:00

As a first time entrepreneur you probably have tons of questions. And every time you do a Google search for an answer you are bombarded with too much information and in some cases that information contradicts other things you have heard. Due to this, I have created a list of 54 resources that should help you out.
Legal & Accounting
Legal and accounting issues may not seem important when you are starting your company, but they are. Legal and accounting mistakes that you make early on can haunt you for years and can be expensive to fix. So if you are going to start a company you should do things right from the get go.
S Corp. vs. LLC: Which Structure is Right for Your Business - Determining the type of legal structure for a new business can be daunting for entrepreneurs and small business owners. Learn more about S Corporations and Limited Liability Companies (LLC), and decide if one of these business structures is right for you.
Legal Zoom – a cheap way to get incorporated.
Findlaw – a directory of all the lawyers throughout the US.
Bookkeeping 101: Debits and Credits - Accounting ends with score keeping but begins with record keeping. The first task of accounting is to accurately record transactions. Transactions are events that change the composition of a firm’s assets, liabilities, and equity.
Accounting Basics - This explanation of accounting basics will introduce you to some basic accounting principles, accounting concepts, and accounting terminology.
Docstoc – A free place to get legal documents and templates which can drastically help reduce your legal fees and in some cases allow you to do some legal stuff yourself.
You’ve Been Sued: What Do You Do? - Everything you wanted to know about being sued.
Opening a Business Bank Account – Business bank accounts and your identity.
Closing Down Your Business Permanently - If you’re shuttering your business for good, there’s more to it than drawing the blinds.
Legal Issues to Consider When Starting Your Business - There are a multitude of legal issues to think about when it comes to starting your business. Everything from your business name to its structure to its operation has legal implications.
Web Design
Design is something we tend to take for granted. Not only is important for your website to look good, but you also want to make sure it is usable and converts.
Top Ten Mistakes in Web Design - The ten most egregious offenses against users. Web design disasters and HTML horrors are legion, though many usability atrocities are less common than they used to be.
Google Website Optimizer - A free tool that will help you maximize your conversions.
Web Developers Handbook - Everything you need to know about web design and development.
CSS Vault - Web design inspiration.
A Roadmap to Web Standards - Before you start designing your website, make sure you are complying with web standards.
Signal vs Noise - A design and usability blog by 37signals.
Crazy Egg - A simple tool that shows you how website visitors interact with your website.
How to choose a website design agency - Choosing the right web design shop can be a tricky thing.
Internet Marketing
You can have a great product or service, but if no one sees it you will never make any money. Now this doesn’t mean you have to hire a marketing firm to help you out, but you could learn some basic things about Internet marketing.
Beginner’s Guide to Search Engine Optimization - The basics of search engine optimization.
Social Media Marketing Beginner’s Guide - Interested in leveraging the social web? Here is what you need to know first.
Top Ten Things You Can Do To Get Blogged On - Getting bloggers to blog on your company isn’t an easy thing. If you flow these rules and you should be successful at it.
How to Optimize Your Blog for Search Engines - What you need to know if you want your corporate blog to get traffic.
Web Analytics: An Hour A Day - Google’s analytics guru, Avinash Kaushik, breaks down web analytics and the tracking needs you may need for your website.
Dullest - The blog of Google’s main search engineer, Matt Cutts. Before you get into search engine optimization, you probably want to read his blog.
MailChimp - Sending high quantities of emails isn’t an easy thing to do. In most cases your emails won’t go through unless you use a service like MailChimp.
Banner Basics: Create Ads That Work - Banner advertising isn’t dead… you just have to learn how to effectively use them.
Hiring Employees
When you don’t have much cash in the bank, you can’t afford to make hiring mistakes. Sooner or later you are going to have to hire a few employees, so you better know what to look for.
When to hire your first employee - if you are considering hiring an employee, you should read this first.
5 tips for hiring your first employee - Microsoft breaks down some useful tips for new companies.
9 Steps Before You Hire - 9 good tips for first time business owners.
ADP - Payroll can be a hassle. ADP provides good payroll solutions for a very affordable price.
How to Fire an Employee - If you hire a bad employee you better learn to fire them.
When to Fire an Employee - Analyzing when it is time to accept that a performance improvement will not happen.
Fully-Baked: The real cost of adding a new employee - Employees cost more than just their salaries.
201 ways to turn any employee into a star performer - A book that you should read if you want to maximize your employees output.
Tsheets - Employee time tracking software.
RescueTime - Employee time tracking and time management software.
Raising Venture Capital
Raising money can be a pain in the ass, especially if you have never done it before. If you want to raise money, you need to know the basic terminology that venture capitalists use, how to create a pitching deck, and how to get in front of venture capitalists.
Vfinance - A directory of venture capitalists, angel investors, and business plan templates.
Forbes Midas List - A list of the top 100 venture capitalists for the year 2009.
The 10/20/30 Rule of PowerPoint - Before you make a powerpoint that showcases your company and how much money you are raising, you should read this.
How to Raise Venture Capital - An detailed overview on how you can raise money.
Venture Hacks - A blog for entrepreneurs that discusses everything about venture capital.
The Funded - An online community of entrepreneurs to research, rate, and review funding sources worldwide.
How To Raise Venture Dollars - Ben Elowitz who has raised over 40 million dollars breaks down the tricks to raising money.
Paul Graham - Awesome essays about venture capital and entrepreneurship.
General Business Advice
Other entrepreneurs have already solved many of the problems you are going to face. So when you run to into these generic problems, here are some websites you can turn to.
Inspired Startup - Successful entreprenuer Andy Liu breaks down how to take your startup to the next level.
The 50 best business blogs - There are thousands of business blogs, but here are the 50 best according to Times.
The Top 10 Mistakes People Make When Starting A Business - You don’t want to make any of these mistakes.
10 Timeless Business Tips From 10 Millionaires - If you want to be a millionaire then you better learn from them.
Solving Your Business Problems - Start solving your problems before they even take place.
Struggling Entrepreneur - The Struggling Entrepreneur is a free podcast series that deals with the issues, challenges and problems of the individual who wants to be an entrepreneur, but who encounters a somewhat difficult path and appears to be struggling.
Smarta - A support community for entrepreneurs.
Litemind - Exploring ways to use your mind more efficiently.
Mixergy - Successful CEOs share their experiences with the masses.
Living The Frugal Life
Although it may sound sexy to be an entrepreneur, most entrepreneurs don’t make a ton of money. You are going to have to learn to live a frugal life so that you can continue to do what you love and not worry about paying your mortgage.
Little Steps: 100 Great Tips For Saving Money For Those Just Getting Started
75 Painless Money-Saving Tips - Saving money can be painful, but here are 75 ways that are painless.
Strapped For Cash? Billshrink Can Save You Thousands!
High Yield Savings Accounts at 20 Largest Banks - If you are going to keep your money with a bank, might as well be a bank that has high yield savings accounts.
Scrooge Strategy - 1 premium tip every week that helps you save money.
How to Reduce Rent - Rent is probably your biggest expense, here are a few ways to reduce it.
This articled is adapted from Neil Partel (quicksprout.com)

High-Performance Web Site

2008-12-12T22:25:00.000-08:00

Frontend Performance Best Practices

1. Make fewer HTTP requests

- Most browsers only download two resources at a time from a given hostname
-Several techniques exist for reducing the number of HTTP requests without reducing
page content:

Combine multiple scripts into a single script.

Combine multiple stylesheets into a single stylesheet.

Combine multiple CSS background images into a single image called a CSS sprite (see http://alistapart.com/articles/sprites).

2. Use a content delivery network (CDN)

A content delivery network (CDN) is a collection of distributed Web servers used to deliver content to users more efficiently. Examples include Akamai Technologies, Limelight Networks, SAVVIS, and Panther Express. The main performance advantage provided by a CDN is delivering static resources from a server that is geographically closer to the end user. Other benefits includebackups, caching, and the ability to better absorb traffic spikes.

3. Add an Expires header

When a user visits a Web page, the browser downloads and caches the page’s resources. The next time the user visits the page, the browser checks to see if any of the resources can be served from its cache, avoiding time-consuming HTTP requests. The browser bases its decision on the resource’s expiration date. If there is an expiration date, and that date is in the future, then the resource is read from disk. If there is no expiration date, or that date is in the past, the browser issues a costly HTTP request. Web developers can attain this performance gain by specifying an explicit expiration date in the future. This is done with the Expires HTTP response header, such as the following:
Expires: Thu, 1 Jan 2015 20:00:00
GMT

4. Gzip components

The amount of data transferred over the network affects response times, especially for users with slow network connections. For decades developers have used compression to reduce the size of files. This same technique can be used for reducing the size of data sent over the Internet. Many Web servers and Web hosting services enable compression of HTML documents by default, but compression shouldn’t stop there. Developers should also compress other types of text responses, such as scripts, stylesheets, XML, JSON, among others. Gzip is the most popular compression technique. It typically reduces data sizes by 70%.

5. Put stylesheets at the top

Stylesheets inform the browser how to format elements in the page. If stylesheets are included lower in the page, the question arises: What should the browser do with elements that it can render before the stylesheet has been downloaded? One answer, used by Internet Explorer, is to delay rendering elements in the page until all stylesheets are downloaded. But this causes the page to appear blank for a longer period of time, giving users the impression that the page is slow. Another answer, used by Firefox, is to render page elements and redraw them later if the stylesheet changes the initial formatting. This causes elements in the page to “flash” when they’re redrawn, which is disruptive to the user. The best answer is to avoid including stylesheets lower in the page, and instead load them in the HEAD of the document.

6. Put scripts at the bottom

External scripts (typically, “.js” files) have a bigger impact on performance
than other resources for two reasons. First, once a browser starts downloading
a script it won’t start any other parallel downloads. Second, the browser won’t
render any elements below a script until the script has finished downloading.
Both of these impacts are felt when scripts are placed near the top of the
page, such as in the HEAD section. Other resources in the page (such as images)
are delayed from being downloaded and elements in the page that already exist
(such as the HTML text in the document itself) aren’t displayed until the earlier
scripts are done. Moving scripts lower in the page avoids these problems.

7. Avoid CSS expressions

CSS expressions are a way to set CSS properties dynamically in Internet Explorer. They enable setting a style’s property based on the result of executing JavaScript code embedded within the style declaration. The issue with CSS expressions is that they are evaluated more frequently than one might expect—potentially thousands of times during a single page load. If the JavaScript code is inefficient it can cause the page to load more slowly.

8. Make JavaScript and CSS external

JavaScript can be added to a page as an inline script
or as an external script:

Similarly, CSS is included as either an inline style block or an external stylesheet. Which is better from a performance perspective? HTML documents typically are not cached because their content is constantly changing. JavaScript and CSS are less dynamic, often not changing for
weeks or months. Inlining JavaScript and CSS results in the same bytes (that haven’t changed) being downloaded on every page view. This has a negative impact on response times and increases the bandwidth used from your data center. For most Web sites, it’s better to serve JavaScript and CSS via external files, while making them cacheable with a far future Expires header as explained in Rule 3.

9. Reduce DNS lookups

The Domain Name System (DNS) is like a phone book: it maps a hostname to an IP address. Hostnames are easier for humans to understand, but the IP address is what browsers need to establish a connection to the Web server. Every hostname that’s used in a Web page must be resolved using DNS. These DNS lookups carry a cost; they can take 20–100 milliseconds each. Therefore, it’s best to reduce the number of unique hostnames used in a Web page.

10. Minify JavaScript

As described in Rule 4, compression is the best way to reduce the size of text files transferred over the Internet. The size of JavaScript can be further reduced by minifying the code. Minification is the process of stripping unneeded characters (comments, tabs, new lines, extra white space, and so on) from the code. Minification typically reduces the size of JavaScript by 20%. External scripts should be minified, but inline scripts also benefit from this size reduction

11. Avoid redirects

Redirects are used to map users from one URL to another. They’re easy to implement and useful when the true URL is too long or complicated for users to remember, or if a URL has changed. The downside is that redirects insert an extra HTTP roundtrip between the user and her content. In many cases, redirects can be avoided with some additional work. If a redirect is truly necessary, make sure to issue it with a far future Expires header (see Rule 3), so that on future visits the user can avoid this delay.

12. Remove duplicate scripts

If an external script is included multiple times in a page, the browser has to parse and execute the same code multiple times. In some cases the browser will request the file multiple times. This is inefficient and causes the page to load more slowly. This obvious mistake would seem uncommon, but in a review of U.S. Web sites it could be found in two of the top 10 sites. Web sites that have a large number of scripts and a large number of developers are most likely to
suffer from this problem.

13. Configure ETags

Entity tags (ETags) are a mechanism used by Web clients and servers to verify that a cached resource is valid. In other words, does the resource (image, script, stylesheet, among others) in the browser’s cache match the one on the server? If so, rather than transmitting the entire file (again), the server simply returns a 304 Not Modified status telling the browser to use its locally cached copy. In HTTP/1.0, validity checks were based on a resource’s Last-Modified date: if
the date of the cached file matched the file on the server, then the validation succeeded. ETags were introduced in HTTP/1.1 to allow for validation schemes based on other information, such as version number and checksum.

ETags don’t come without a cost. They add extra headers to HTTP requests and responses. The default ETag syntax used in Apache and IIS makes it likely that the validation will erroneously fail if the Web site is hosted on multiple servers. These costs impact performance, making pages slower and increasing the load on Web servers. This is an unnecessary loss of performance,
because most Web sites don’t take advantage of the advanced features of ETags, relying instead on the Last- Modified date as the means of validation. By default, ETags are enabled in popular Web servers (including Apache and IIS). If your Web site doesn’t utilize ETags, it’s best to turn them off in your Web server. In Apache, this is done by simply adding “FileETag none” to your
configuration file

14. Make Ajax cacheable

Many popular Web sites are moving to Web 2.0 and have begun incorporating Ajax. Ajax requests involve fetching data that is often dynamic, personalized, or both. In the Web 1.0 world, this data is served by the user going to a specified URL and getting back an HTML document. Because the HTML document’s URL is fi xed (bookmarked, linked to, and so on), it’s necessary to ensure the response is not cached by the browser. This is not the case for Ajax responses. The URL of the Ajax request is included inside the HTML document; it’s not bookmarked or linked to. Developers have the freedom to change the Ajax request’s URL when they generate the page. This allows developers to make Ajax responses cacheable. If an updated version of the Ajax data is available, the cached version is avoided by adding a dynamic variable to the Ajax URL. For example, an Ajax request for the user’s address book could include the time it was last edited as a parameter in the URL, “&edit=1218050433.” As long as the user hasn’t edited their address book, the previously cached Ajax response can continue to be used, making for a faster page.

SOurce:

This is a short version of " High-Performance Web Site" article published in ACM communication issue 12 by Steve Souders

Steve Souders (http://stevesouders.com) works at Google on Web performance and open source initiatives. He is the author of High Performance Web Sites and the creator of YS low, Cuzillion, and Hammerhead. He teaches at Stanford and is the co-founder of the Firebug Working Group

Quantum Cryptography

2008-12-10T17:50:00.000-08:00

Key to the quantum industry

Technology that exploits the strange rules of quantum mechanics to guarantee the security of encrypted messages is the first product of a new quantum-information industry to reach the market, as Andrew Shields and Zhiliang Yuan explain

As theories go, quantum mechanics has certainly been successful. Despite its many counterintuitive predictions, it has provided an accurate description of the atomic world for more than 80 years. It has also been an essential tool for designing today's computer chips and hard-disk drives, as well as the lasers used in the fibre-optic communications of the Internet. Now, however, the ability to manipulate the quantum states of individual subatomic particles is allowing us to exploit the strange properties of quantum theory much more directly in information technology.
Key to the quantum industry
Key to the quantum industry

We are used to thinking of information as being abstract, but in fact all information requires a physical medium for its processing, storage and communication. The basic unit of information – a bit that is either "0" or "1" – can be represented physically by, for example, the current in a circuit or light in an optical fibre. As information is represented by ever smaller physical systems, quantum effects become increasingly important. The ultimate limit comes when bits are represented by the quantum state of a single particle, such as the polarization of a photon.

Applied to information, quantum theory throws up some very odd predictions. These are not only interesting as a test of quantum mechanics, but can also bring us practical applications that are simply impossible with "classical" information technology. For example, a quantum computer would work with bits that can be both "0" and "1" at the same time, allowing it to solve certain mathematical problems – such as factorizing very large numbers – that are virtually intractable using an ordinary computer.

Although practical quantum computers will take many years to develop, one manifestation of quantum information technology is already a reality: quantum cryptography. This ultra-secure way of sending messages is based on the fundamental postulate that measuring a quantum state will, in general, alter it. Thus, if we encode messages in individual quantum states, such as the phase of photons whizzing down an optical fibre, an eavesdropper who tries to intercept the message cannot avoid changing it. We can therefore test if the message has been read before it reaches the intended recipient – something that is impossible using classical signals.

The commercial potential of quantum cryptography has attracted private investment in several start-up companies in the US and Europe. The firm id Quantique, for example, spun out from pioneering research at the University of Geneva; while in the US, commercial developments are led by MagiQ Technologies, based in New York and Massachusetts. Recently a third start-up called SmartQuantum has been established in Brittany, France, and major corporate players such as HP, IBM, Mitsubishi, NEC, NTT and Toshiba all have active quantum-cryptography programmes. With several quantum-cryptography products already on the market, the quantum information industry has arrived.
The key to security

Cryptography is a vital part of today's computer and communication networks, protecting everything from business e-mails to bank transactions and Internet shopping. Information is generally kept secret using a mathematical formula called an encryption algorithm, together with a secret "key" that the sender uses to scramble a message into a form that cannot be understood by an eavesdropper. The recipient then uses the same key – typically a long binary number – with a decryption algorithm to read the message.

Although modern algorithms such as the Advanced Encryption Standard (AES) are very hard to break without the key, this system suffers from an obvious weakness: the key must be known to both parties. Thus the problem of confidential communication reduces to that of how to distribute these keys securely – the encrypted message itself can then safely be sent along a public channel (figure 1). A common method is to use a trusted courier to transport the key from sender to receiver.
Figure 1
Figure 1

However, any distribution method that relies on humans is vulnerable to the key being revealed voluntarily or under coercion. In contrast, quantum cryptography, or more accurately quantum key distribution (QKD), provides an automated method for distributing secret keys using standard communication fibres. The revolutionary feature of QKD is that it is inherently secure: assuming that the laws of quantum theory are correct, we can prove that the key cannot be obtained by an eavesdropper without the sender and recipient's knowledge. Furthermore, QKD allows the key to be changed frequently, reducing the threat of key theft or "cryptanalysis", whereby an eavesdropper analyses patterns in the encrypted messages in order to deduce the secret key.

The first method for distributing secret keys encoded in quantum states was proposed in 1984 by theoretical physicists Charles Bennett at IBM and Gilles Brassard at the University of Montreal. In their "BB84" protocol, a bit of information is represented by the polarization state of a single photon – "0" by horizontal and "1" by vertical, for example. The sender (Alice) transmits a string of polarized single photons to the receiver (Bob) and by carrying out a series of quantum measurements and public communications they are able to establish a shared key and to test whether an eavesdropper (Eve) has intercepted any bits of this key en route.

The BB84 protocol allows us not only to test for eavesdropping, but also to guarantee that Alice and Bob can establish a secret key even if Eve has determined some of the bits in their shared binary sequence, using a technique called "privacy amplification". Imagine, for example, that Eve knows 10% of the key bits shared by Alice and Bob. Being aware of this, Alice and Bob could then publicly agree to add together (using modular arithmetic) each adjacent pair of bits to form a new sequence of half the length. Eve may also do this, but since she will need to know both bits in a pair in order to correctly determine their sum, she will find that she now shares a much lower fraction of the new bit sequence with Alice and Bob.

So much for the principle. In practice, generating the pulses of single photons required for BB84 is not easy. Despite recent progress using single atoms or semiconductor quantum dots to generate single photons (see Physics World February 2003 "Single photons on demand"), most practical QKD systems use weak laser pulses to send the bits that make up the key. This method has an Achilles heel: the laser will sometimes generate pulses containing two or more photons, each of which will be in the same quantum state. As a result, Eve could split off one of these photons and measure it, while leaving the other photons in the pulse undisturbed, thus determining part of the key while remaining undetected. Even worse, by blocking the single-photon pulses and allowing only the multi-photon pulses to travel through to Bob, Eve could determine the entire key.

Until true single-photon sources become available commercially, the most common defence is to strongly attenuate the laser to limit the rate of multi-photon pulses. However, this also means that many pulses contain no photons at all, reducing the rate at which the key can be transmitted. In 2003 a new trick to get round this problem was proposed by Hoi-Kwong Lo at the University of Toronto and Xiang-Bin Wang at the Quantum Computation and Information Project in Tokyo, based on earlier work by Won-Young Hwang at Northwestern University in the US.

Their idea was to intersperse the signal pulses randomly with some "decoy pulses" that are weaker on average and so very rarely contain a multi-photon pulse. If Eve attempts a pulse-splitting attack, she will therefore transmit a lower fraction of the decoy pulses to Bob than the signal pulses. Thus by monitoring the transmission of the decoy and signal pulses separately, Eve's attack can be detected. This means that stronger laser pulses may be used securely – for instance, last year at Toshiba we demonstrated a 100-fold increase in the rate that keys can be transmitted securely over a 25 km fibre. The decoy-pulse protocol has caused great excitement in the QKD community, with four independent groups having just reported experimental demonstrations of the technique.

Weak laser pulses are not the only way to carry out quantum cryptography. For example, QKD using a true single-photon source has recently been demonstrated at Stanford University, the CNRS in Orsay and Toshiba. Furthermore, in 1991 Artur Ekert, while a PhD student at the University of Oxford, described an alternative to the BB84 protocol that exploits another counterintuitive prediction of quantum mechanics: entanglement. Pairs of entangled photons have quantum states that are strongly correlated, such that measuring one photon affects the measurement of the other. If Alice and Bob each have one of the pair, they can therefore use their measurements to exchange information. This technique has been demonstrated by researchers at the University of Vienna, the Los Alamos National Laboratory and the University of Geneva, and was even used in 2004 to transfer money between Vienna City Hall and an Austrian bank. However, weak-laser QKD is the most mature approach, and the basis of the commercial QKD systems that are now coming on the market.
Practical QKD

Information can be encoded in the quantum state of photons in several different ways. The first laboratory demonstration of QKD by Bennett and Brassard in 1989 over 30 cm of air used the polarization state of photons. However, transmitting photons along an optical fibre can randomize their polarization, so a better approach pioneered by Paul Townsend, formerly of BT Labs in the UK, is to alter the phase of the photon. In this method, weak laser pulses are injected into an interferometer by Alice. By applying different voltages to a "phase modulator" in one arm of the interferometer, Alice can encode bits as a phase difference between the two emergent pulses sent to Bob – for example with 0° representing "0" and 180° representing "1". Bob then passes the pulses through another interferometer and determines which of his two detectors, corresponding to "0" and "1", they emerge at (see figure 2).
Figure 2
Figure 2

For this scheme to work, we must keep the relative lengths of the interfering paths in Alice and Bob's interferometers stable to a few tens of nanometres. However, temperature changes of just a fraction of a degree are enough to upset this balance. An ingenious solution to this problem was introduced by the Geneva group in 1997, which led to the first QKD system suitable for use outside the lab. The idea is to send the laser pulses on a round trip from Bob to Alice and then back to Bob so that any changes in the relative arm lengths are cancelled out. A QKD system based on this design is currently available for about €100,000 from the University of Geneva spin-out company id Quantique.

At the Toshiba lab in Cambridge, we have developed an alternative compensation technique that allows pulses to be sent just one way, by sending an unmodulated reference pulse along with each signal pulse. These reference pulses are used as a feedback signal to a device that physically stretches the fibre in one of the two arms of the interferometer to compensate for any temperature-induced changes. In trials with the network operator Verizon, the one-way QKD system was continuously operated for over a month without requiring any manual adjustment.

We can assess the performance of QKD systems by the rate at which secure bits can be exchanged. The faster the secure-bit rate, the more frequently the key can be changed, thus inhibiting cryptanalysis. Typical secure-bit rates for complete QKD systems are in the range 10–50 kbit s–1 for a 20 km fibre link. Although this may seem low compared with the rate data are transferred in optical communications (typically 1–40 Gbit s–1), it is enough for up to 200 AES encryption keys (each of which comprises 256 bits) to be sent per second – sufficient for most cryptographic applications.

The secure-bit rate that can be achieved decreases with the length of the optical link due to the scattering of photons from the fibre. For this reason, the best performance is usually achieved using photons with a wavelength of 1.55 µm, at which standard optical fibre is most transparent. Even so, when the fibres get so long that the signal rate becomes comparable to the rate of false counts in Bob's photon detector, sending a secure key is no longer possible. For the standard indium gallium arsenide (InGaAs) semiconductor detectors used to detect 1.55 µm photons, this distance is currently about 120 km. Recently the Los Alamos group has used low-noise superconducting detectors to extend secure key distribution to fibres 150 km in length. Significantly, these distances are long enough for almost all the spans found in today's fibre networks.

Although the risk of cryptanalysis is mitigated by using QKD to frequently refresh the encryption key, it is not eliminated entirely. However, this can be achieved by encrypting the message using a "one-time pad", which requires a random key that contains the same number of bits as the message. Each bit of the message is then encrypted by adding it to the corresponding bit in the key using modular arithmetic. Provided that the key distribution is unconditionally secure, as it is using QKD, and that the key is never reused, the one-time pad is completely immune to attack. The downside is the length of the key that must be exchanged. QKD bit rates are already sufficient to allow unconditionally secret voice communication using the one-time pad. In the future, higher bit rates will allow this security to be extended to other forms of data.

Today's secure-bit rates are limited by how often the InGaAs detectors can detect a photon currently once every 100 ns. Silicon-based photon detectors can operate almost 1000 times faster, but they are only sensitive to shorter-wavelength photons. As the quality of InGaAs detectors improves over the next few years, we can expect their frequency to catch up with that of silicon, leading to QKD bit rates that are orders of magnitude higher. In the interim, there are encouraging results showing that non-linear crystals may be used to shift 1.55 µm photons to shorter wavelengths for which the faster silicon detectors may be used. Higher detection rates have also been demonstrated using superconducting nanowire detectors, and recent advances with detectors based on quantum dots are also encouraging.
Towards a quantum network

One of the first real-life applications of QKD has been to secure fibre links between corporate sites in a city. Companies are increasingly using high-bandwidth optical connections between offices, data centres, server farms and disaster-recovery sites to obtain the speed and convenience of a local area network over a larger geographical area. In the early days of fibre deployment, immunity to "tapping" of sensitive data was often cited as a key advantage of fibre over copper cable. But in fact, eavesdropping on optical fibres can be accomplished by simply introducing a small bend in the fibre to extract a portion of the light; and, in the absence of quantum cryptography, it is almost impossible to detect.

At Toshiba, we have developed a "link encryptor" that can send data at 1 Gb s–1 between corporate sites, combining AES data encryption with secure key distribution using one-way QKD (figure 3). Meanwhile, id Quantique announced that it will install its "Vectis" link encryptor between the two centres of data-hosting company IX Europe in Zurich. In the US, MagiQ Technologies has recently developed its own encrypted link, targeted at government applications including the military, intelligence gathering and homeland security.
Figure 3
Figure 3

An important next step will be extending QKD from single point-to-point links into a "quantum network" for key distribution. Networks allow a company to connect multiple sites securely and to add new sites for an incremental cost. Moreover, they allow the range of QKD to be increased from the length of a single fibre link to any distance covered by the network, and safeguard against outages of individual links by automatically routing traffic around them.

In October 2003 BBN Technologies set up a primitive but pioneering QKD network in Cambridge, Massachusetts, linking their site with Harvard and Boston Universities. The firm showed that it was possible to direct the stream of single photons between different receiving units using an optical switch, and it also introduced the idea of "key relay" along a chain of trusted nodes. Here, each pair of adjacent nodes in the chain stores its own local key. A global key may then be sent from one end of the chain to the other, over any distance, by using the local keys and a one-time pad to encrypt each hop.

A more sophisticated system is currently under development by the European SECOQC consortium, a collaboration of academic and industrial QKD researchers, classical cryptographers and telecoms engineers. It is developing the protocols required for routing, storage and management of keys within a meshed network that could in principle be very large. A trial implementation of the quantum net is planned in 2008 that will allow any two users at several sites across Vienna to establish a shared key.

These QKD networks assume that the intermediate nodes are secure, which is realistic if the network is operated by a single service provider. In the future, however, we can relax this condition using a device called a "quantum repeater". Quantum repeaters are based on the principle of quantum "teleportation", whereby a quantum state is transferred from one location to another, in principle over an arbitrary distance, using a pair of entangled particles. Recent developments such as a semiconductor device for generating entangled photon pairs and the teleportation of quantum states between photons and atoms bring the quantum repeater closer to becoming a reality.

Meanwhile, an alternative to using a fibre-optic network to send quantum keys over long distance may be to use free-space links to low-orbit communication satellites. In 2006 a collaboration between researchers at the universities of Vienna, Munich and Bristol implemented a free-space link over 144 km between Tenerife and La Palma.
Selling quantum cryptography

From the first laboratory demonstrations over 30 cm of air to the latest fibre-based systems operating over 100 km, QKD has certainly come a long way in the last two decades. The technology has shrunk into compact units the size of typical network equipment and is fully automated. But despite the technical progress there are significant barriers to the adoption of new cryptographic technologies.

A particular problem for QKD is selling technology based on quantum mechanics to clients who often know little about physics and are used to traditional cryptography. Another hurdle is the lack of a security certification process for the equipment. Users need reassurance not only that QKD is theoretically sound, but also that it has been securely implemented by the vendors. It is encouraging that there are several initiatives under way to establish common security standards for QKD.

As the market for QKD develops, we can expect that the price of equipment will drop significantly. Within 10 years we may see QKD used not only in corporate and government networks, but also in networks serving home users. Optical fibres are already used to deliver television, phone and Internet services to domestic users in several countries. Although current QKD systems are too expensive for such applications, they may become viable if miniaturization to microchip-scale and mass-production lead to the expected price reductions. The days when the products of the quantum-information industry serve every household may not be too distant.
The BB84 protocol
The BB84 protocol
The BB84 protocol

Quantum cryptography is a way of generating a shared secret key that can be used to encrypt and decrypt messages, for example by encoding information in the polarization state of individual photons. In the BB84 protocol, the sender (Alice) transmits photons to the recipient (Bob) in one of four different polarization states: horizontal (H), vertical (V), diagonal (D, 45°) and anti-diagonal (A, –45°). For each photon she sends, Alice randomly selects one of these polarizations, with H or D representing the bit value "0" (red) and V or A representing "1" (blue), depending on the "basis" she chooses. To measure the photons, Bob is equipped with an analyser that can distinguish either between H and V (+) or between A and D (×). He randomly (and independently from Alice) chooses which analyser he will use to measure each photon. If Bob selects the analyser that is compatible with Alice's choice (top), he will determine the photon's polarization, and thus the bit value, with certainty. If, on the other hand, Bob measures with the "wrong" analyser (middle), he will obtain a random result.

It seems problematic that half of Bob's measurements result in a random bit value. However, Alice and Bob have a cunning solution. After Bob's measurements have taken place, he reveals the sequence of analysers that he used. Alice then tells him which times he used the correct analyser, without revealing the bit that she sent. They can then discard all the measurements for which Bob used the wrong analyser, ensuring that they share the same bit sequence without any errors (in the absence of noise or imperfections).

This post-selection leaves an eavesdropper (Eve) at a disadvantage since she must guess which analyser to use to measure each photon (bottom). Inevitably Eve will sometimes select an analyser that is incompatible with Alice's choice of polarization, and thus may obtain a result that differs from the bit Alice sent. The key to the secrecy of quantum cryptography is that by making this measurement, Eve inevitably changes the quantum state of the photon. Therefore, when Bob receives the photon, he will sometimes determine an erroneous bit value even when he and Alice used compatible measurements. By examining a small sample of their bit sequence for errors, Alice and Bob can therefore determine whether an eavesdropper was present.
At a Glance: Quantum cryptography

* The quantum states of individual subatomic particles can be used to encode information, opening up applications in communication and computing
* The most mature application of quantum theory to information processing is quantum cryptography, with products already on the market
* Quantum cryptography, also known as quantum key distribution, allows us to send encrypted messages the secrecy of which can be guaranteed by allowing an eavesdropper to be detected
* Secure messages have been sent over distances in excess of 100 km using quantum cryptography with photons carried by optical fibres
* The next step will be to establish a "quantum network" that could allow quantum cryptography to cover cities and eventually the globe

.htaccess File

2008-12-09T23:30:00.000-08:00

Stronger Security

We set the access on your public_html directory so that the files inside public_html could be read by the web server, in fact the files are quite exposed. It is possible for other students to read the contents of your directory and steal your work.

When you use the chmod command to allow the web server access to files in your public_html directory, you are also allowing other students to view the files in your public_html directory as well. This is because the chmod command treats the web server user ("wwwsrv") and other students usernames as the same (i.e. they both fall into the "other" category for chmod).

What you really want to do is just allow the web server to be able to read your public_html directory and its contents, not all of the other students. New versions of Unix provide a mechanism called ACLs (Access Control Lists) to allow you to give permissions for specific users to access files/directories.
Protecting your Files - ACLs (Access Control Lists) and .htaccess

This procedure to set up acl protection is entirely optional. You don't have to do it if you don't want to. However, it is recommended that you do it.

There is a supplementary mechanism for protecting html files involving a file called .htaccess. The .htaccess procedure acts to protect your files and should also be used as well as setfacl. An outline of how to use .htaccess protection is detailed in the next section.
If you want your files to be completely protected you need acl and htaccess protection
The easiest way to set up ACL Protection

ACLs in their native form are quite complex. At FIT we have created a program that makes using ACLs easier.

(a) First clear all ACLs on your home directory and public_html.
"c" means "clear"
"r" means "recursive"

acl -c ~
acl -cr ~/public_html

(b) Set the normal Unix permissions back to protected/safe ones
using the chmod command:

chmod 700 ~
chmod -R 700 ~/public_html

(c) Allow the "wwwsrv" user only to access your public_html
directory, and don't allow it to write to your public_html
directory either, only read/execute ("rx" permission).
"a" means "add"
"r" means "recursive"

acl -a wwwsrv=rx ~
acl -r -a wwwsrv=rx ~/public_html

(d) Check the ACL and it should show that the wwwsrv user has
read and execute permission, the user/owner has all permissions,
but the group and others have none.
If you provide no parameters to the "acl" command it just
shows you what the current ACL is.

acl ~
acl ~/public_html

For protecting your work, particularly when doing assignments that require creating web pages, the best approach is to use chmod to make sure all access for other people are removed, then use ACLs to selectively grant access to the files/directories you want to make public (and to specific users like 'wwwsrv').

Note that when you use ACLs to protect your web content, it means that the other users cannot open your files by changing directory into your public_html directory and opening them up in a text editor. However it does not stop other students from viewing your pages on the web, through a browser.
Setfacl - An alternative way to set up ACL protection

The procedure will not alter the visibility of your pages to web browsers, but it will mean that the contents of protected directories will not be able to be read by normal command line unix procedures by anyone other than the owner of the files. It's a better arrangement than the existing one.

OK here's what you need to do to set up setfacl:

(1) logon to your account on charlie using ssh. (from the FIT Labs, work, or home it doesn't matter).

(2) Decide which directories and files in your public_html directory you want to protect. Let's say for the sake of example that you want to protect them all. (That's probably the best policy)

(3) run the command:

chmod 700 public_html

This will change the access to your public_html directory to owner can read, write and execute, but no-one else can do anything. So your files are completely inaccessible to anyone other than you.

(4) The Apache web Server on charlie needs access however. This can be done by running the command:-

setfacl -r -m u:wwwsrv:r-x public_html

This allows the user wwwsrv to also have access to the public_html directory as well as the files inside. wwwsrv is the system user for the Apache Web server running on charlie.

Note also that the permissions of the files and directories inside public_html don't have to be changed.

(5) Check that you can still access the web pages inside your public_html directory by pointing a browser at your home pages.

OK, that's all you have to do. If you are really interested in what's happening you can have a look at the man page for setfacl on charlie. An alternative way of setting up acl protection is outlined below. This alternative method was provided by Dr Wayne Brookes.
Setting up htaccess protection

If you have implemented the acl mechanism correctly, then your public_html directory cannot be accessed by other users from the command line. However, there is still a problem. Users can point their web browser at your home page and view your source html by using the view source option on the browser. One way to plug this security hole is to use the .htaccess mechanism.

You have to create two files. One in the directory that you want to restrict access to. This file is called .htaccess (Note the full stop at the start of the filename - it's important). So if you wanted to restrict access to your public_html directory the .htaccess file will be in your public_html directory.

The format of the file is :-

AuthName "Enter user name and password for access"
AuthUserFile /home/jsmith/misc/.allowed.users
AuthType Basic
require user test xyz abc

Please note that in the above .htaccess file, there are three authorised users. These are test, xyz and abc. There is no user called "user".

The last line shows the login names of the users that you are allowing to access the dirctory. The AuthUserFile directive specifies the location and name of the file containing the passwords of the three authorised users test, xyz and abc. In the example above the password file is .allowed.users and it's in the directory /home/jsmith/misc/ directory. You have to create this password file. This can be done easily by typing in :-

cd
mkdir misc
cd misc
touch .allowed.users

Once you have created this file you have to add encrypted passwords. You can do this by using the utility htpasswd.

You execute this command with 2 arguments. These are the path and name of the file to contain the passwords and the name of the user you want to add to the file. So, to continue our example, you would enter :

htpasswd /home/jsmith/misc/.allowed.users test

This will prompt you for the password for test. You can add in the other
users abc and xyx by substituting abc and xyz for test when you call
htpasswd.

The last thing to do is set the permissions on .htaccess and .allowed.users. You can do this by using the chmod 644 .allowed.users and chmod 644 .htaccess You should also make sure you chmod 755 the misc directory.

UTS short Course

2008-12-09T18:16:00.000-08:00

Java Programming Advanced (J2EE)

Advanced Java Programming teaches some of the foundation skills required for building medium to large scale web-based applications, with a B2B e-commerce focus.

For further information and registration please click here or contact Francoise Ware on 9514 2154 or email fware@it.uts.edu.au

My Career

2008-12-09T05:14:00.000-08:00

Apature

2008-12-02T03:16:00.000-08:00

Link: http://www.apture.com

I came across the video about new web service which use wikipedia and other webservice to serve an article.

I'm gonna use it for vds-nsw.com

tuanbkit@gmail.com
Pw: MyHoney09

Code Spelunking: Exploring Cavernous Code Bases

2008-11-15T00:39:00.001-08:00

by George V. Neville-Neil, Consultant

Code diving through unfamiliar source bases is something we do far more often than write new code from scratch--make sure you have the right gear for the job.

Try to remember your first day at your first software job. Do you recall what you were asked to do, after the human resources people were done with you? Were you asked to write a piece of fresh code? Probably not. It is far more likely that you were asked to fix a bug, or several, and to try to understand a large, poorly documented collection of source code.

Of course, this doesn't just happen to new graduates; it happens to all of us whenever we start a new job or look at a new piece of code. With experience we all develop a set of techniques for working with large, unfamiliar source bases. This is what I call code spelunking.

Code spelunking is very different from other engineering practices because it is done long after the initial design and implementation of a system. It is a set of forensic techniques used after the crime has been committed.

There are several questions that code spelunkers need to ask, and tools are available to help them answer these questions. I will look at some of these tools, addressing their shortcomings and pointing out possible improvements.

SIZE DOES MATTER

Source bases are already large, and getting larger all the time. Table 1 shows the sizes of some popular open source software. The Linux kernel, with support for 17 different processor architectures, is made up of 642 directories, 12,417 files, and more than 5 million lines of code. A complex network server such as Apache is made up of 28 directories and 471 files--encompassing over 158,000 lines of code--whereas an application such as the nvi editor contains 29 directories, 265 files, and over 77,000 lines of code. I believe that these examples are fairly honest representations of what people are confronted with when they start a job.

Of course, even larger systems exist for scientific, military, and financial applications, but those shown in table 1 are more familiar and should help convey an instinctive feeling for the complexity involved in systems that we all come in contact with every day.

Unfortunately, the tools we have to work with often lag behind the code we are trying to explore. There are several reasons for this, but the primary one is that very few companies have been able to build a thriving business on tools. It is much easier to make money selling software that appeals to a broad audience, which software tools do not.

Static vs. dynamic. We can use two scales to compare code spelunking tools and techniques. The first one ranges from static analysis at one end to dynamic analysis at the other. In static analysis you're not observing a running program but are examining only the source code. A clear example is using the tools find, grep, and wc to give you a better idea of the size of the source code (as was done to produce table 1). A code review is an example of a static technique. You print the code, sit down at a table with it, and begin to read.

On the dynamic end of the scale are tools such as debuggers. Here you run the code on real data, using one program (the debugger) to examine another as it executes. Another dynamic tool is a software oscilloscope, which displays a multithreaded program as a series of horizontal time lines--like those on a real oscilloscope--to find deadlocks, priority inversions, and other bugs common to multithreaded programs. Oscilloscopes are used mostly in embedded systems.

Brute force vs. subtlety. The second type of scale measures the level of finesse that is applied to spelunking. At one extreme is a brute-force approach, which often consumes a lot of CPU time or which might generate a large amount of data. An example of brute force is attempting to find a bug by using grep to locate a message that prints out near to where the error occurs.

Table 1 Sizes of Popular Software
Program	Version	Directories	Files	Lines
Apache Web Server	1.3	28	471	158,332
DB	1.4.25	176	234	99,836
Emacs	21	43	2586	1,317,915
FreeBSD Kernel	5.1	420	4758	2,140,517
Linux Kernel	2.4.20-8	642	12,417	5,223,290
Nvi	1.81.5	29	265	77,176
Python	2.2.3	245	1158	356,314
Note: Every attempt was made to disregard documentation or other files not directly related to the source code.

At the other end of the finesse scale is subtlety. A subtle approach to finding a string within a program involves a tool that builds a database of all the interesting components of a code base (e.g., function name, structure definitions, and the like).You then use that tool to generate a fresh database each time you update your source from the code repository. You would also use it when you want to know something about the source, as it already has the information you need at its virtual fingertips.

Plotting your method. You can use both of these scales to create a two-dimensional graph in which the x-axis represents finesse and the y-axis runs from static to dynamic. You can then plot tools and techniques on this graph so that they can be compared (see figure 1). None of the terms used implies value judgments on the techniques. Sometimes a static, brute-force approach is just what you want. If it would take just as long (or longer) to set up a tool to do some subtle analysis, a brute-force approach makes sense. You don't often get points for style when code spelunking; people just want results.

One thing to keep in mind when code spelunking is the Scout motto, "Be prepared." A little effort spent up front getting your tools in place will always save time in the long run. After a while, all engineers develop a stable of tools that they use to solve problems. I have a few tools that I always keep on hand when code spelunking, and I make sure to prepare the ground with them whenever I start a project. These tools include Cscope and global, which I'll discuss in more detail later.

A very common scenario in code spelunking is attempting to fix a bug in an unfamiliar piece of code. Debugging is a highly focused task: You have a program, it runs, but not correctly. You must find out why it does this, where it does this, and then repair it. What's wrong with the program is usually your only known quantity. Finding the needle buried in the haystack is your job, so the first question must be, "Where does the program make a mistake?"

You can approach the problem in several ways, many of which are shown in figure 1. The approach you choose depends on the situation. If the program is a single file, you could probably find the bug through inspection, but as table 1 demonstrates, any truly useful application is much larger than a single file.

Let's take a theoretical example. Jack takes a job with the Whizzo Company, which produces WhizzoWatcher, a media player application that can play and decode many types of entertainment content. On his first day at work (after he has signed up for health insurance, the stock plan, and 401K) Jack's boss e-mails him two bug reports to deal with.

The two bugs that Jack has just been assigned are as follows:

Bug 1: When WhizzoWatcher opens a file of type X it immediately crashes with no output except a core file.
Bug 2: While watching a long movie on DVD (The Lord of the Rings, The Two Towers) the audio sync is lost after about two hours. This does not happen at any particular frame; it varies.

WhizzoWatcher 1.0 is a typically organic piece of software. Originally conceived as a "prototype," it wowed the VPs and investors and was immediately rushed into production over the objections of the engineers who had written it. It has little or no design documentation, and what exists is generally inaccurate and out of date. The only real source of information on the program is the code itself. Because this was a prototype, several pieces of open source software were integrated into the larger whole and were expected to be replaced when the "real system" was funded. The total system now consists of about 500 files that spread over 15 directories, some of which were written in-house and some of which were integrated.

Bug 1. This is the easiest bug for Jack to work on; the program crashes when it starts. He can run it in the debugger, and the offending line will be found at the next crash. In terms of code spelunking, he doesn't have to look at much of the code at first, although becoming generally familiar with the code base will help him in the long run.

Unfortunately, Jack finds that although the reason for the immediate crash is obvious, what caused it is not. A common cause of crashes in C code is dereferencing a null pointer. At this point in the debugger, Jack doesn't have the previous states of the program, only the state at the moment it crashed, which is actually a very small amount of data. A common technique is to visually inspect the code while stepping up the stack trace to see if some caller stomped on the pointer.

A debugger that could step backward, even over a small number of instructions, would be a boon in this situation. On entry into a function the debugger would create enough storage for all of the function's local variables and the application's global variables so that they could be recorded, statement by statement, throughout the function. When the debugger stopped (or the program crashed), it would be possible to step backward up to the start of the function to find out which statement caused the error. For now, Jack is left to read the code and hope to stumble across the real cause of the error.

Bug 2. Attacking Bug 2, where the code doesn't crash but only produces incorrect results, is more difficult because there isn't a trivial way to get a debugger to show you when to stop the program and inspect it. If Jack's debugger supports conditional breakpoints and watchpoints, then these are his next line of defense. A watchpoint or a conditional breakpoint tells the debugger to stop when something untoward happens to a variable, and allows Jack to inspect the code at a point closest to where the problem occurs.

Once Jack has found the problem, it's time to fix it. The key is to fix the bug without breaking anything else in the system. A thorough round of testing is one way to address this problem, but he'll feel more comfortable making a fix if he can find out more about what it affects in the system. Jack's next question ought to be, "Which routines call the routine I want to fix?"

Attempting to answer this question with the debugger will not work, because Jack cannot tell from the debugger all the sources that will call the offending routine. This is where a subtle, static approach bears fruit. The tool Cscope builds a database out of a chunk of code and allows him to do the following:

Find a C symbol
Find a global definition
Find functions called by a function
Find functions calling a function
Find a text string
Change a text string
Find an egrep pattern
Find a file
Find all files that #include this file

You will note that item 4, "Find functions calling a function," answers his question. If the routine he's about to fix modifies nonlocal variables or structures, he must next answer the question, "With which functions does my function share data?" This would never come up in a "properly designed program" (i.e., one written from scratch). Jack would never use a horde of global variables to control his program, because he knows what a nightmare it would be to maintain. Right?

Of course, this is code spelunking, so Jack is already past that point. Using a subtle tool such as Cscope again is tempting, but this turns out to be a case where brute force is his best hope. Generating a list of all the global references in a program (file name and line number) is certainly possible for a compiler, but none of them do this. Although this option would hardly be used in the creation of new code, it would make the process of debugging old code far easier. Jack will have to make do with a combination of find and grep to figure out where all these variables reside in the program.

Code spelunking isn't something you do only when debugging; it's how you perform a good code review, reverse-engineer a design document, and conduct a security audit.

During an age when many people use computers for financial and personal transactions, auditing code for security holes has (or should) become commonplace. In order to close these security holes, you have to know what the common attacks are, as well as which sections of the code are vulnerable to attack. Although the attacks are updated almost daily on the Bugtraq mailing list (http://www.securityfocus.com), what we're concerned with is finding them.

Consider the following example. Jill takes a job with a large bank that serves most of its customers electronically, over the Internet. On her first day her boss tells her that the bank is doing a security audit of its entire system, which is implemented on several different types of hardware and operating systems. One set of Unix servers handles incoming Web traffic and then makes requests to a mainframe backend that actually manages the real money.

One possible security hole occurs when a program stores a user's private data in a way that makes it available to anyone with access to the machine. An example of this is storing a password as plain text in a file or registry key. If Jill had written the software, or had access to the person who did, she could quickly find out where the program stores passwords simply by asking. Unfortunately, the person who wrote the login scripts was laid off when the bank moved its headquarters six months ago. Jill will have to find out how this is done without the author's help.

Unlike in the debugging case, few tools are available to tell Jill something as specific as, "Where does the program store data X?" Depending on the structure of the program, this could happen anywhere--and often does. Jill can tackle this problem with either a brute-force or a subtle approach. To do the former she would litter the code with debugging statements (i.e., printfs or language equivalent) to find out where the program stores the password. She probably has a few guesses to narrow the field from the entire program down to a few or a dozen locations, but this is still a good deal of work.

A more subtle approach would be to run the program in the debugger, attempt to stop the program right after the user enters a password, and then follow the execution until the program performs the storage operation.

A typical way to attack a program is to give it bad input. To find these vulnerabilities, Jill must ask the question, "Where does the program read data from an untrustworthy source?" Most people would immediately think that any code dealing with network data would be vulnerable, and they would be right--in part. With the advent of networked file systems, the fact that the code is executing a read() statement does not imply that it is reading from a local (i.e., trustable) source.

Whereas Jack's debugging in my earlier example attempted to zero in on a problem, Jill's code audit is more of a "fan-out." She wants to see as much of the code as possible and understand how it interacts both with its own modules ("How is data passed around?") and with external entities ("Where do we read and write data?"). This can be an even more daunting task than finding a needle in a haystack, and may be more like labeling all the individual straws. In this case, the most important thing for Jill to do is to find the most likely places that will cause problems--that is, the places that are executed most often.

There is a tool that, although not originally intended for this purpose, can help to focus Jill's efforts: a code profiler. For example, gprof was originally written to tell an engineer which routines within the program are using up all the CPU time and are, therefore, candidates for optimization. The program is run with a workload (let a user bang on it or have it service requests from the network), and then the output is analyzed. A profiler will tell Jill which routines are being called most often. These are obviously the routines to check first. There is no reason for Jill to pore over the large portion of the code that is called infrequently while the routines called most often may have gaping holes.

Routines that pass bad arguments to system calls are another common security problem. This is most often exploited against network servers in an effort to gain control over a remote machine. Several commercial tools attempt to discover these kinds of problems. A quick and dirty approach is to use a system call tracer, such as ktrace or truss, to make a record of what system calls are executed and what their arguments are. This can give you a good idea of where possible errors lie.

Code spelunking is about asking questions. The challenge is to get your fingers around the right parts of the code and find the answers without having to look at every line (which is a near impossibility anyway). There is one tool I haven't yet mentioned in this article, and that's the one sitting inside your head. You can begin applying good engineering practices even though they weren't used to create the code you're spelunking.

Keeping a journal of notes on what you've found and how things work will help you to create, and keep in mind, a picture of how the software you're exploring works. Drawing pictures is also a great help; these should be stored electronically along with your notes. Good experiment design of the type you may have learned--and then promptly forgotten--in physics class is also helpful. Just beating on a piece of code until it works is not an efficient way of figuring it out. Setting up an experiment to see why the code behaves a certain way may take a lot of thought, but usually very little code.

Finally, one of my favorite tools is the "stupid programmer trick." You get a colleague to look at the code and then you attempt to explain to him or her what the code does. Nine times out of ten your colleague won't even need to say anything. Fairly quickly you realize what's going on, smack yourself on the forehead, say thank you, and go back to work. Through the process of systematically explaining the code out loud, you have figured out the problem.

There is no one tool that will make understanding large code bases easier, but I hope that I've shown you a few ways to approach this task. I suspect you'll find more on your own.

Global

This is the tool I apply to every source base that I can. Global is really a pair of tools: gtags and htags. The first, gtags, builds a database of interesting connections based on a source tree in C, C++, Java, or Yacc. Once the database is built, you can use your editor (both Emacs and vi are supported) to jump around within the source. Want to know where the function you're calling is defined? Just jump to it. The second tool is htags, which takes the source code and the database generated by gtags and creates an HTML-browsable version of the source code in a subdirectory. This means that, even if you don't use Emacs or vi, you can easily jump around the source code finding interesting connections. Building the database is relatively speedy, even for large code bases, and should be done each time you update your sources from your source-code control system.

Cscope

Cscope was originally written at AT&T Bell Labs in the 1970s. It answers many questions, such as: Where is this symbol? Where is something globally defined? Where are all the functions called by this function? Where are all the functions that call this function? Like Global, it first builds a database from your source code. You can then use a command-line tool, Emacs, or one of a few GUIs written to work with the system to get your questions answered.

gprof

This is the standard profiling tool on most open source Unix systems. Its output is a list of routines ordered by how often they were called and how much of the program's CPU time was spent executing them. This can be useful in figuring out where to start looking for security holes in a program.

ktrace

This is a standard tool on open source operating systems. The name stands for "kernel trace." It will give you a listing of all the system calls made by a program. The output includes the arguments and return values given to and received from the calls. You use ktrace by running a program under it and then dumping the output with kdump. It is available on all open source Unix operating systems.

truss

This is available only on Solaris. It is the Solaris version of ktrace--basically the same tool, with different arguments.

code spelunking

This Web site contains a TWiki collaboration page dedicated to code spelunking.

Link: Code Spelunking