Computers are inherently discrete -- it uses discrete mathematics to do discrete computations before displaying an image over a finite number of pixels. However, the scenes we are sending our raytracer (e.g., spheres) are continuous. At its root, we are attempting to force a higher-frequency signal into a lower-frequency representation. To do this, we sample the original signal, typically along some kind of rectangular grid (or in two or more dimensions using some less axis-aligned representation (see Lanczos resampling).

Anti-aliasing attempts to reduce some of the side-effects of forcing higher-frequency details into a low frequency representation (see attached images, source: Wikipedia). Note the artifacting present at the top of the first image. For more information, check out a graphics book or signal processing book!

Implementing basic antialiasing in our raytracing is pretty simple. Instead of just sampling along a regular n x n grid and averaging the resulting colors to give our final color, we first jitter each ray in the grid on random. This modicum of random noise added to each ray further prevents aliasing.

// Sample along a grid . . .
for(aa_i=0; aa_i<aa_factor; aa_i++)
{
for(aa_j=0; aa_j<aa_factor; aa_j++)
{
    // Add a random epsilon to the ray
    double jitter_i = (aa_i + (double) rand()
        / std::numeric_limits<unsigned int>::max()) * aa_recip;
    double jitter_j = (aa_j + (double) rand()
        / std::numeric_limits<unsigned int>::max()) * aa_recip;

    Ray* ray = camera->getRay( (i+0.5 + jitter_i)/width - 0.5,
        (j+0.5 + jitter_j)/height - 0.5 );
    ...
    final_color += computed_color * weighting_matrix[aa_i][aa_j];
}}

Note the use of a weight matrix above. We sample more heavily toward the middle of the pixel, paying less attention to the edges. See the code below for an example matrix.

// Weight matrix for 4x AA
double weighting_matrix[4][4] = {{ 0.03125, 0.0625, 0.0625, 0.03125},
    { 0.0625, 0.125, 0.125, 0.0625},
    { 0.0625, 0.125, 0.125, 0.0625},
    { 0.03125, 0.0625, 0.0625, 0.03125}};

No Antialiasing Enabled

4x Antialiasing Enabled