On Wed, 25 Mar 2015 15:48:54 -0700, Razer
I called the police on a "Nicotine Nazi" one evening in my California town after he stuffed a camera in my face and called me a 'criminal' (if he only knew...) for ROLLING (not smoking) a cigarette on a Starbucks patio and was informed by the police it IS legal to take 'portrait photos' without the subject's permission (assuming they stay out of arms reach), and further (in case you were wondering why bullying is so prevalent) that you can say anything you want short of threats of violence and it IS NOT "assault".
Was the Starbucks Patio visible from a 'public' area, like the sidewalk? Then one could argue you don't have much of an expectation of privacy. If I was in this position and the Nicotine Nazi refused to stop filming after being politely asked to do so, I would pick up and move inside the restaurant, somewhere out of camera shot. If they followed you in and kept filming I would ask the manager to escort them off the premises.