The traditional HOG algorithm extracts pedestrian features from whole image, a large number of non-human window calculation is bound to reduce the accuracy and efficiency of detection. In this connection, a pedestrian detection and tracking method based on OTSU segmentation and HOG feature was proposed. The image was segmented by the OTSU algorithm with the best threshold value, on the basis of the segmentation region, the image contour could be generated through canny edge detection, and methods of applying symmetry to calculate the image edge could determine human candidate region. Then combining HOG features after PCA dimensionality reduction with Hidden Markov Model to detect and verify pedestrian candidate region. Finally, taking determined pedestrian area as the tracking window to complete tracking pedestrian by using CamShift algorithm. Several experiments results prove that the efficiency and accuracy of pedestrian detection were improved by the method of this paper, and its tracking performance was stable and reliable.