Most online instances of "European Projects Office", or at least a large proportion of them, seem to refer to Spain, raising the obvious suspicions that they have been translated from "Oficina de Proyectos Europeos". That doesn't mean that the expression is wrong, but it does mean that these instances can't be taken as evidence that it is idiomatic in English. It may well have been translated by someone who is not a native speaker of English, and even if the translator was a native speaker of English, many translators lack the ability to detach themselves from the source term and judge what would really be said in English.

As a general principle, nouns used adjectivally in English are normally put in the singular even if their sense is plural and they would be plural in the equivalent Spanish expression. An estación de autobuses is a bus station, not a buses station. When I revise texts in English written by Spanish speakers with a good command of English, I often find that they have pluralised preposed adjectival nouns that should naturally be singular.

And yet there are many exceptions: many cases in which the preposed adjectival noun is naturally plural. Universities have Admissions Offices, not Admission Offices, for example. The reasons why the plural may be preferred are various and often subtle, and I won't attempt to account for them in general terms here, but there is no doubt that offices dealing with European projects can naturally be, and are, called European Projects Offices, though they can be, and are, also called European Project Offices.

Here are a couple of undoubtedly native English examples in which there has clearly been no interference from Spanish or any other foreign language:

"Europe Services PMO Lead at NCR Corporation
London, Greater London, United Kingdom
[...]
Leader of NCR's European Project Office. We deliver multinational technology projects and transitions for our customers"
https://uk.linkedin.com/in/christian-keyes-190a3454

"Hewlett-Packard Ltd,
European Projects Office,
Filton Road,
Stoke Gifford,
BRISTOL BS34 8QZ,
UK"
http://www0.cs.ucl.ac.uk/research/m3i/annex1.html

"The administrative arm of ESDN remains on the Manchester campus, but has relocated to the European Projects Office, housed within the purpose built North West Genetics Knowledge Park (Nowgen Centre)."
http://www.esdn.org/eug/digitalAssets/0/260_ESDNnewsletter7J...

Another instance of a European Project Office responsible for European Projects is Claire's first reference.

However, Claire's second reference illustrates the fact that the singular and plural forms can have different meanings. It refers to the "European Project Office" (and "North American Project Office") of a single project (the Baseline ALMA Bilateral Project).
http://jump2.nrao.edu/dbtw-wpd/textbase/Documents/brown-Trip...

And if we remove "European", a "project office" will generally be understood as an office dealing with a single project; an office dealing with projects in general will tend to be called a projects office.

That is why I say that in my view, although both are certainly correct, there is an argument for using the plural when it is an office dealing with projects rather than a project. There's certainly nothing wrong with the plural here. Personally I would stay with it.