Will all peppers turn red?

A year ago, I got a couple of pepper plants from a student-run farmer's market, and two of them are just labeled "small chili peppers". The plants are thriving in my window sill this year (they produced like one pepper each last year) and there are a few peppers nearing six inches long, bright green.

I picked and ate one at about four inches and it was good, but I'd like to let them ripen if they will. How can you tell when a pepper is ripe? Will it change color? Are the green peppers I see at stores just less ripe?