Background: The purpose of this study was to examine associations between perceptions of neighborhood safety and physical activity among youth. Methods: We completed a cross-sectional study of children age 8 to 9 years (n = 188) and adolescents age 13 to 15 years (n = 346) in areas of varying socioeconomic status in Melbourne, Australia. Parents and adolescents completed questionnaires on perceptions of neighborhood safety. Scores were computed for perceptions of road safety, incivilities, and personal safety of the child or adolescent. Moderate-to-vigorous physical activity (MVPA) before or after school, on evenings, and on weekends was recorded using accelerometers. Results: There were no associations between parental perceptions of neighborhood safety and children’s MVPA outside school hours. Parental perception of personal safety was positively associated with adolescent boys’ MVPA after school. Adolescent girls’ concern about road safety was negatively associated with their MVPA during evenings and outside school hours. Conclusion: Perceptions of neighborhood safety might influence physical activity among youth in different ways according to age group and sex.