President Barack Obama on Friday staunchly defended the sweeping U.S. government surveillance of Americans’ phone and internet activity, calling it a “modest encroachment” on privacy that was necessary to defend the United States from attack.

“Nobody is listening to your telephone calls. That’s not what this program is about,” Obama told reporters during a visit to California’s Silicon Valley. He emphasized that the secret surveillance programs were supervised by federal judges and authorized by Congress, which had been briefed on the details.

Obama’s comments came after reports this week in Britain’s Guardian newspaper and the Washington Post revealed that the National Security Agency and the FBI had secretly conducted surveillance of Americans’ telephone and internet communications activities far beyond what had been made public.

The reports triggered a broad debate about privacy rights and the proper limits of government surveillance in the aftermath of the September 11, 2001, attacks on the United States. They also sent White House officials and congressional leaders scrambling to explain why the government needs to collect information on trillions of phone calls and internet communications.

“In the abstract you can complain about Big Brother and how this is a potential program run amok, but when you actually look at the details, I think we’ve struck the right balance,” Obama said, noting that a secret federal court reviews requests for surveillance and that Congress is briefed on such activity.

He acknowledged having “a healthy skepticism” about the programs before he was first elected in 2008, but that he had since come to the conclusion that such “modest encroachments on privacy” were worth it.

“You can’t have 100 percent security and also then have 100 percent privacy and zero inconvenience,” Obama said. “We’re going to have to make some choices as a society. … There are trade-offs involved.”