The measurement error resulting from graduation anomalies and the signal processing algorithm used for determining the positions of graduations on line scales was investigated by simulation and experiment. Optical image-forming simulations were carried out on models of 6-µm-wide graduations with three sizes of defects (0.5, 1.0 and 1.5 µm) at one edge. A digital filter was used in signal processing to obtain the first differential to determine the positions of the graduations. The minimum values of the lateral shift of the determined graduation positions were observed for the three defect sizes when using a 9-µm-wide differential filter. An experiment was also carried out on an ordinary line scale with 6-µm-wide graduations using a high-precision laser-interferometric line scale calibration system by measuring seven positions on the scale in the direction perpendicular to the measurement axis. The root mean square of the standard deviations from the linear fitting lines constructed using the measured positions over a 300-mm-long line scale was 2.8 nmwhen the differential filter width was 9 µm. It was demonstrated that a differential filter was effective in reducing the lateral error due to graduation anomalies.

1. [1] M. A. Jimarez, S. Tran, C. L. Coz, and G. O. Dearing, “Evolution of a Unique Flip-Chip MCM-L Package,” IEEE Trans. Advanced Packaging, Vol.22, pp. 372-378, 1999.
   http://dx.doi.org/10.1109/6040.784488
2. [2] A. Lassila, E. Konen, and K. Riski, “Interferometer for calibration of graduated line scales with a moving CCD camera as a line detector,” Appl. Opt., Vol.33, pp. 3600-3603, 1994.
   http://dx.doi.org/10.1364/AO.33.003600
3. [3] J. S. Beers and W. B. Penzes, “The NIST length scale interferometer,” J. Res. Natl. Inst. Stand. Technol., 104, 225-252, 1999.
4. [4] T. B. Eom and J.W. Han, “A precision length measuring system for a variety of linear artifacts,” Meas. Sci. Technol., Vol.12, pp. 698-701, 2001.
   http://dx.doi.org/10.1088/0957-0233/12/6/307
5. [5] E. F. Howick and C. M. Sutton, “Development of an automatic line scale measuring instrument,” Proc. SPIE, Vol.4401, pp. 112-119, 2001.
   http://dx.doi.org/10.1117/12.445611
6. [6] I. Fujima, Y. Fujimoto, K. Sasaki, H. Yoshimori, S. Iwasaki, S. Telada, and H. Matsumoto, “Laser interferometer for calibration of a line scale module with analog output,” Proc. SPIE, Vol.5190, pp. 103-110, 2003.
   http://dx.doi.org/10.1117/12.508118
7. [7] J. Flügge, C. Weichert, H. Hu, R. Köning, H. Bosse, A. Wiegmann, M. Schulz, C. Elster, and R. D. Geckeler, “Interferometry at the PTB Nanometer Comparator: design, status and development,” Proc. SPIE, Vol.7133, 713346, 2009.
   http://dx.doi.org/10.1117/12.821252
8. [8] A. A. Bolonin, “New VNIIM comparator for measurements of line scales for length,” Meas. Techniques, Vol.50, pp. 503-508, 2007.
   http://dx.doi.org/10.1007/s11018-007-0100-9
9. [9] H. Gonzalez, C. Galvan, and J. A. Muñoz, “Image processing automatic interferometric calibration system for line scales,” Proc. SPIE, Vol.5190, pp. 93-102, 2003.
   http://dx.doi.org/10.1117/12.503865
10. [10] S. Kaušinis, A. Jakštas, R. Barauskas, and A. Kasparaitis, “Investigation of dynamic properties of line scale calibration systems,” XVIII IMEKOWorld Congress Metrology for a Sustainable Development, 2006 September 17-22, Rio de Janeiro, Brazil, 2006.
11. [11] G. Hermann, “Calibration machine for line scales,” IEEE 7th Int. Symp. on Intelligent Systems and Informatics, pp. 227-239, 2009.
    http://dx.doi.org/10.1109/SISY.2009.5291158
12. [12] W. Israel, I. Tiemann, G. Metz, Y. Yamaryo, F. Maeda, and T. Shimomura, “An international length comparison at an industrial level using a photoelectric incremental encoder as transfer standard,” Precis. Eng. Vol.27, pp. 151-156, 2003.
    http://dx.doi.org/10.1016/S0141-6359(02)00192-7
13. [13] M. Sawabe, F. Maeda, Y. Yamaryo, T. Simomura, Y. Saruki, T. Kubo, H. Sakai, and S. Aoyagi, “A new vacuum interferometric comparator for calibrating the fine linear encoders and scales,” Precis. Eng. Vol.28, pp. 320-328, 2004.
    http://dx.doi.org/10.1016/j.precisioneng.2003.11.007
14. [14] A. Takahashi and N. Miwa, “An experimental verification of the compensation of length change of line scales caused by ambient air pressure,” Meas. Sci. Technol., Vol.21, 045305, 2010.
    http://dx.doi.org/10.1088/0957-0233/21/4/045305
15. [15] J. Flügge, R. Schödel, R. Köning, and H. Bosse, “Long term stability of Suprasil line scales and gauge blocks,” Proc. 10th Int. Conf. of European Society for Precision Engineering and Nanotechnology, Delft, 262V1, 2010.
16. [16] A. Takahashi, “Long-term dimensional stability and longitudinal uniformity of line scales made of glass ceramics,” Meas. Sci. Technol., Vol.21, 105301, 2010.
    http://dx.doi.org/10.1088/0957-0233/21/10/105301
17. [17] M. Druzovec, B. Acko, A. Godina, and T. Welzer, “Simulation of line scale contamination in calibration uncertainty model,” Int. J. Simul. Model., Vol.7, pp. 113-123, 2008.
    http://www.ijsimm.com/Abstracts/Abstracts7-3.pdf accessed on 5 October, 2011
18. [18] M. Družovec, B. Ačko, A. Godina, and T. Welzer, “Robust algorithm for determining line centre within a video positional measuring system,” Opt. Lasers Eng., Vol.47, pp. 1131-1138, 2009.
    http://dx.doi.org/10.1016/j.optlaseng.2009.06.017
19. [19] J. A. Muñoz-Gómez, and C. Galvan, “Robust line detection for line scale calibration,” Advanced mathematical & computational tools in metrology & testing VIII, World Scientific Publishing Company, p. 243, 2009, accessed on 5 October, 2011.
    http://uqu.edu.sa/files2/tiny_mce/plugins/filemanager/files/4281670/tomas/tomas7/sspss/book_matl/book_matl/book_matl2/probability/pro1/applied_2/
    appliedpro_2/progzi/azkaar/9812839518.pdf
20. [20] H. Wei, W. Wang, G. Ren, and L. Pei, “Algorithm for determining line centre with microscope measuring system,” Proc. SPIE, Vol.7855, 785524, 2010.
    http://dx.doi.org/10.1117/12.871046
21. [21] J. Flügge, R. Köning and H. Bosse, “Recent activities at PTB nanometer comparator,” Proc. SPIE, Vol.5190, pp. 391-399, 2003.
    http://dx.doi.org/10.1117/12.506908
22. [22] R. Köning, J. Flügge, and H. Bosse, “Achievement of sub nanometer reproducibility in line scale measurements with the Nanometer Comparator,” Proc. SPIE, Vol.6518, 65183F, 2007.
    http://dx.doi.org/10.1117/12.712121
23. [23] M. Sawabe, F. Maeda, Y. Yamaryo, T. Shimomura, Y. Saruki, H. Sakai, and S. Aoyagi, “Development of the vacuum interferometric comparator for calibrating fine linear encoders and scales,” Proc. SPIE, Vol.4900, pp. 282-289, 2002.
    http://dx.doi.org/10.1117/12.484567
24. [24] M. Born and E. Wolf, “Principle of optics,” 7th (Ed.), p. 598, Cambridge University Press, 1999.
25. [25] A. Takahashi, Y. Takigawa, and N. Miwa, “Error contributor of defocus and quadratic caustic in line scale measurement,” Meas. Sci. Technol., Vol.22, 015302, 2011.
    http://iopscience.iop.org/0957-0233/22/1/015302/
26. [26] A. Savitzky and M. J. E. Golay, “Smoothing and differentiation of data by simplified least squares procedure,” Anal. Chem., Vol.36, pp. 1627-1639, 1964.
    http://dx.doi.org/10.1021/ac60214a047
27. [27] Manufacturer’s web page, Optenso, accessed on 5 October, 2011.
    http://www.optenso.com/index.html
28. [28] Manufacturer’s Specification, 5517B, Agilent Technologies, accessed 5 October, 2011.http://cp.literature.agilent.com/litweb/pdf/5964-6190E.pdf
29. [29] T. J. Quinn, “Practical realization of the definition of the metre,” including recommended radiations of other optical frequency standards, 2001, Metrologia, Vol.40, pp. 103-133, 2003.
    http://dx.doi.org/10.1088/0026-1394/40/2/316
30. [30] J. Ishikawa, “Portable national length standards designed and constructed using commercially available parts,” English edition 2, Synthesiology, pp. 246-257, 2010, accessed on 5 October, 2011.
    http://www.aist.go.jp/aist_e/research_results/publications/synthesiology_e/vol2_no4/vol2_04_p246_p257.pdf
31. [31] “Manufacturer’s Specification,” Zerodur, Schott AG, accessed on 5 October, 2011.http://www.schott.com/advanced_optics/english/our_products/zerodur/zerodur.html
32. [32] R. Köning, B. Przebierala, C. Weichert, J. Flügge and H. Bosse, “A revised treatment of the influence of the sample support on the measurement of line scales and the consequences for its use to disseminate the unit of length,” Metrologia, Vol.46, pp. 187-195, 2009.
    http://dx.doi.org/10.1088/0026-1394/46/3/005