In our managed package, we make heavy use of Dynamic SOQL so that we can optimize our queries to be as efficient as possible at runtime. We use Dynamic SOQL to do things like limiting the fields that are queried based on what features are enabled. All of these things help us to optimize our application and the overall Salesforce user experience for our mutual customers.
When running in the context of the managed package, dynamic SOQL doesn't require you to add a namespace to query objects in the package. This is because there is some behind-the-scenes magic that automatically injects the namespace into the queries if the name of the object that is being queried matches a name of an object in the package. This is meant to be a developer convenience but actually causes some serious issues.
This is problematic when our application is installed into a customer's organization that happens to have an object with the same API name as an object in our package. If our application tries to query our own object, the Dynamic SOQL "assumes" that we are trying to query the customer's object of the same name because of what's called "Shadowing". In this scenario, the namespace is not injected resulting in the query running against the customer's object. With no way to overried the namespace injection behavior, it essentially makes it impossible to reach our own object with a Dynamic SOQL query in this scenario.
This is listed as a "No Fix" on the known issues: https://success.salesforce.com/issues_view?id=a1p30000000T3V2AAK
The issue also occurs at the field level when fields of the same name are added. See here: https://help.salesforce.com/apex/HTViewSolution?id=000228097&language=en_US
The suggested workaround is to have our customers re-name their objects and/or fields. This is not really a viable solution, nor is it the right solution long-term.
I propose one of the following, non-breaking API additions to Dynamic SOQL that would allow applications at runtime to skip shadowing, or reverse the effects of shadowing.
Option 1: Overload Database.query() with an additional boolean flag that skips shadowing
// Database.query(String query, Boolean allowShadowing)
List<SObject> records = Database.query('SELECT Id FROM Foo__c', false); // returns package Foo__c object
Option 2: Add an Additional Database Method to handle forcing of the namespace-injected queries
// Database.queryWithoutShadowing(String query);
List<SObject> records = Database.queryWithoutShadowing('SELECT Id FROM Foo__c'); // returns package Foo__c object
Implementing either of these two options would give managed package developers a tool they can use to make sure that package objects and fields are always accessible from Dynamic SOQL.