Can the police do this?

I was taking a walk at night with two of my friends and we were smoking weed in the park. After we were done and walking away, a police car pulled up out of nowhere and asked how we were doing and if we were smoking weed. We told him that we were he then proceeded to search all of us and didn't find anything illegal. he took down our names in the notebook and said that a girl was assaulted at knife point by a "brother" on a bmx bike the previous day and thats the reason for frisking us. He then reiterated that if we have any "brothers" or know any "brothers" with a bmx bike we should contact him. My question is could we have refused the search? Were the allowed to frisk us like that?