Stop “Zooming,” Start Sampling: How to Respect Nyquist on the OLS 5000

You’re in the metrology lab. Your task: perform a critical failure analysis on a new silicon-carbide (SiC) substrate using the lab's flagship, the Olympus OLS 5000 laser scanning microscope. The goal is to quantify the average surface roughness, or R_a, to see if it meets the 5 nm spec.

You mount the sample, find a wide area with the 10x objective, and switch to the 100x for a high-resolution look. You locate a representative region and—as decades of computer use have taught you—your instinct takes over. You drag a “zoom” box over the feature of interest and crank the digital zoom up to 8× for a really close look.

You hit the Acquire button.

A 3D profile appears... but something is deeply wrong. The topography is jagged, blocky, and aliased. It looks less like a precision-engineered surface and more like a level from Minecraft. The system dutifully reports an R_a of 50 nm. Your high-end atomic force microscope (AFM) data from yesterday said 5 nm.

You sigh, frustrated. You start to troubleshoot. Is the 0.5 nm Z-resolution failing? Is the 405 nm laser unstable? Are there vibrations in the room?

You’re blaming the wrong axis.

Your measurement didn't fail in the Z-direction (height); it failed in the XY-direction (laterally). The culprit isn’t the instrument’s world-class vertical precision. The culprit is a fundamental, avoidable violation of digital sampling theory. The "zoom" feature you just used is a software illusion, and it has turned your quantitative, six-figure-instrument data into a numerical fiction.

Section 2: The Two Non-Negotiable Pillars of Optical Metrology

To produce a 3D surface map that is quantitatively valid (i.e., measurable), you must satisfy two independent but complementary physical principles. They are not suggestions; they are laws.

1. The Abbe Diffraction Limit: This law defines the smallest feature your optics can possibly resolve.
2. The Nyquist-Shannon Sampling Theorem: This theorem defines how your detector must record that feature to reconstruct it faithfully.

Think of it this way: Abbe tells you the size of the smallest "brushstroke" your microscope can paint. Nyquist tells you how close together those brushstrokes must be to create a coherent picture instead of a mess of disconnected dots.

You must satisfy both. Ignoring either compromises your data, your analysis, and your conclusions.

Pillar 1: The Abbe Limit (What You Can See)

The OLS 5000 is a light microscope. Its ability to distinguish two tiny, adjacent points is limited by the wave nature of light itself. This is called diffraction. The smallest lateral distance your microscope can ever resolve is defined by Ernst Abbe’s diffraction limit:

Let’s break down these terms:

• d (distance): This is your minimum resolvable distance, also known as your optical resolution. This is the "size" of your laser spot, often called the Point Spread Function (PSF).
• λ (lambda): This is the wavelength of your light source. The OLS 5000 uses a 405 nm blue-violet laser. This short wavelength is a key advantage, as a smaller λ results in a smaller d (better resolution) compared to red-laser or white-light systems.
• NA (Numerical Aperture): This is the most important specification of your objective lens. Think of it as the size of the "cone" of light the lens can gather. A higher NA (a wider cone) gathers more light and more diffraction information, leading to a much smaller d.

Example: Your 100× Objective

Let's run the numbers for the standard high-resolution 100× objective (MPLAPON100XLEXT):

• λ = 405 nm
• NA = 0.95

This 213 nm is your physical "speed limit." It is the smallest brushstroke your 100x objective can "paint." You cannot see, resolve, or measure any feature smaller than this, period. Your job as a metrologist is not to beat this number—that's impossible—but to sample it correctly.

Pillar 2: The Nyquist Limit (How You Must Record It)

Now that you know your smallest resolvable feature is 213 nm, how do you record it?

This is where the Nyquist-Shannon sampling theorem comes in. In simple terms, the theorem states that to accurately reconstruct a signal (in your case, the topography of your surface), you must sample it at a frequency at least twice as high as the highest frequency in the signal.

Let's use a simpler analogy: The Picket Fence Problem.

Imagine trying to measure the spacing of a picket fence where the posts are 213 cm apart. If you only take one measurement (one "pixel") every 300 cm, you'll miss some posts entirely. When you connect your dots, you'll reconstruct a "fence" with posts 300 cm apart, or worse, a wavy line. You have created aliasing—a false, low-frequency signal—because you undersampled.

To guarantee you capture every post, you must take at least two samples for every post. If your posts are 213 cm apart, you must sample at least every 106.5 cm.

From Fences to Metrology

• Picket Fence Post = Your 213 nm resolvable feature (d)
• Your Sample Spacing = Your Pixel Size

Therefore, the Nyquist criterion for microscopy is:

However, for quantitative surface metrology and accurate 3D reconstruction, a stricter rule is industry standard. The bare minimum of 2x sampling can detect the feature, but it's not enough to accurately measure its shape (e.g., its width, peak, and valley). For that, a "safety factor" is preferred.

Let's recalculate our "magic number":

This 85 nm/pixel is your Golden Rule for the 100x objective. If your scan's pixel size is larger than this (e.g., 120 nm/pixel), you are undersampling and your data is invalid. If it's much smaller (e.g., 30 nm/pixel), you are oversampling—you gain no new spatial information, but you will get larger files and much longer scan times.

Section 3: The "Liar's Zoom" vs. The "Scientist's Scan"

This brings us back to the lab. How do you control your pixel size on the OLS 5000? It's a simple, critical equation:

You have two controls to change this:

1. Number of Pixels (Scan Format): This is your detector array size, e.g., 512×512, 1024×1024, or 2048×2048.
2. Field of View (FOV): This is the physical area the laser scans. On the OLS 5000, this is set by your Optical Zoom (a setting that physically narrows the laser's scan angle).

And then there's the trap:

• Digital Zoom (The "Zoom" Icon 🔍): This button does not change the physics of the scan. It does not change the FOV or the Number of Pixels. It is a post-processing magnifying glass that only stretches the image you've already taken.

Section 4: Your Practical Framework: The OLS 5000 "Nyquist Cheat Sheet"

This entire process can be simplified. You don't need to do the Abbe calculation every time. We've done it for you. Here are the "Golden Rules" for the most common LEXT dedicated objectives on the OLS 5000.

Pin this table to the microscope.

(Note: The 50x and 100x LEXT objectives have the same 0.95 NA, and thus the same theoretical resolution. The 100x provides higher magnification, fitting that 213 nm-resolved detail into a smaller FOV).

The Pro-Metrologist's Workflow

Use this simple checklist every single time you acquire a quantitative scan.

1. Select Objective: (e.g., 50x).
2. Consult Cheat Sheet: Your Nyquist Target is ≤ 85 nm/pixel.
3. Frame Your Feature: Use the Optical Zoom controls to draw a box just large enough to contain your region of interest.
4. Check Your Pixel Size: Look at the OLS 5000 software interface. It will tell you the exact nm/pixel value for your current settings (FOV and Scan Format).
5. Compare and Acquire:
   • If `nm/pixel > 85`: You are undersampling.
     Fix 1 (Fast): Increase Optical Zoom (to shrink FOV).
     Fix 2 (Slow): Increase Scan Format (e.g., 1024 → 2048).
   • If `nm/pixel ≤ 85`: You are correctly sampled. You are clear to hit Acquire.
   • If `nm/pixel << 85`: (e.g., 30 nm/pixel). You are oversampling. This is fine for accuracy, but you are wasting time. Consider decreasing your Scan Format (e.g., 2048 → 1024) to speed up the scan.

Section 5: Why This Matters: The Real-World Cost of Bad Sampling

Undersampling isn't just an "aesthetics" problem. It fundamentally corrupts the physics of your measurement.

• R_a Inflation: This is the most common error. As in our example, the roughness algorithm measures the artificial stair-steps of your blocky pixels instead of the real, nanoscale surface topography. This error can easily inflate your R_a values by 5x, 10x, or more, leading to false part rejections and chasing manufacturing problems that don't exist.
• Feature Misrepresentation (Aliasing): You can't measure what you can't see. If you are trying to measure a 300 nm-wide lithography line with 125 nm/pixel sampling, your measurement is built on only two data points. The edge-detection algorithm will fail. By sampling correctly at 83 nm/pixel, you now measure that same line with three to four data points, enabling a valid, sub-pixel measurement of its critical dimension (CD).
• 3D Reconstruction Failure: A 3D model built from undersampled data is a cartoon. You cannot perform valid failure analysis—like measuring the angle of a micro-fracture or the volume of a pit—on a dataset that looks like a stack of blocks.

Be a Metrologist, Not Just an Imager

A laser confocal profiler is not a point-and-shoot camera. It is a quantitative instrument governed by the same non-negotiable laws of signal processing that underlie all of modern science.

Digital Zoom is for looking. Optical Zoom and Scan Format are for measuring.

Failing to respect the Nyquist limit is the single most common, and most easily avoidable, source of error in 3D optical metrology. It is the equivalent of trying to measure a milligram with a bathroom scale.

Your R_a value, your failure analysis report, your publication, and your scientific reputation depend on this. Before you ever hit that "Acquire" button, ask yourself one question:

Technical Appendix: The Physics of Sampling

A brief reference for the underlying principles.

Abbe Limit (Lateral Resolution)

Defines the minimum resolvable feature size (d) based on laser wavelength (λ) and numerical aperture (NA).

Nyquist-Shannon Sampling Criterion

States that the sampling frequency (f_s) must be greater than twice the maximum signal frequency (f_m). In imaging terms, your pixel size must be less than half the size of the smallest feature you want to resolve (d).

Metrology Standard

For accurate surface metrology and 3D reconstruction, a more robust sampling of 2.3–3 pixels per resolvable feature (d) is recommended (Pawley, 2006). This provides sufficient data points to accurately define the shape of the feature.

Useful vs. Empty Magnification

• Useful Magnification: Any increase in resolution that reduces pixel size to ≤ the Nyquist target. This is achieved via Optical Zoom or Scan Format and gathers new spatial information.
• Empty Magnification: Any digital enlargement of already-acquired data. This is Digital Zoom. It adds no new spatial information and, if the underlying data is undersampled, only serves to visualize interpolation artifacts.

References

1. Jonkman, J. et al. (2020). Guidance for quantitative confocal microscopy. Nature Protocols, 15, 1585–1611.
2. Pawley, J. B. (ed.) (2006). Handbook of Biological Confocal Microscopy, 3rd ed. Springer.
3. NIST/ASME B46.1 – Standard for Surface Texture and Roughness Measurement.