Oracle REST Data Services running on Tomcat - Basic Authentication using JNDI Realm

What do we want to achieve?

We want to protect our REST endpoints using Basic Authentication and authenticate the requests against our users directory (LDAP). We also want to manage the privileges centrally, through the ORDS Roles and Privileges (https://oracle-base.com/articles/misc/oracle-rest-data-services-ords-au…), so no matter if ORDS runs on Oracle WebLogic or Apache Tomcat, it should behave the same and give access to the same resources.

Setup

You should have the following installed:

• ORDS version >=18.1.1
• Tomcat >=8.0.0
• JDK 7 or higher

For me, a great guide to get started quickly is this one for running ORDS in Docker, written by Tim Hall.

First try

In the beginning the problem seems pretty straightforward. We'll try to authenticate using Tomcat Users, following this guide: https://oracle-base.com/articles/misc/oracle-rest-data-services-ords-au…. First, create tomcat-users.xml in "$CATALINA_BASE/conf/tomcat-users.xml" :

<?xml version="1.0" encoding="UTF-8"?>
        <tomcat-users xmlns="http://tomcat.apache.org/xml"
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      xsi:schemaLocation="http://tomcat.apache.org/xml tomcat-users.xsd"
                      version="1.0">
      
          <role rolename="ords-rest-access"/>
          <user username="tomcat" password="tomcat" roles="ords-rest-access"/>
        </tomcat-users>

Next, configure the realm, let's define it in the "$CATALINA_BASE/conf/server.xml" . For more information, see https://tomcat.apache.org/tomcat-9.0-doc/realm-howto.html#Configuring_a…

        <Resource name="UserDatabase" auth="Container"
        type="org.apache.catalina.UserDatabase"
        description="User database that can be updated and saved"
        factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
        pathname="conf/tomcat-users.xml" />
        ...
    <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
    resourceName="UserDatabase"/>

We'll also define and secure an endpoint. It will print for us all the CGI variables and will be accessible only for members of group "ords-rest-access".

 BEGIN
     <!--define endpoint-->
  ORDS.DEFINE_MODULE(
    p_module_name    => 'api.v1',
    p_base_path      => '/api/v1/',
    p_items_per_page =>  25,
    p_status         => 'PUBLISHED',
    p_comments       => NULL);  
        ORDS.ENABLE_SCHEMA(
            p_enabled             => TRUE,
            p_schema              => 'ORDSEXAMPLE',
            p_url_mapping_type    => 'BASE_PATH',
            p_url_mapping_pattern => 'test',
            p_auto_rest_auth      => FALSE);    
      
        ORDS.DEFINE_TEMPLATE(
            p_module_name    => 'api.v1',
            p_pattern        => 'test_endpoint',
            p_priority       => 0,
            p_etag_type      => 'HASH',
            p_etag_query     => NULL,
            p_comments       => NULL);
        ORDS.DEFINE_HANDLER(
            p_module_name    => 'api.v1',
            p_pattern        => 'test_endpoint',
            p_method         => 'GET',
            p_source_type    => 'plsql/block',
            p_items_per_page =>  5,
            p_mimes_allowed  => '',
            p_comments       => NULL,
            p_source         =>
      'BEGIN
      owa_util.print_cgi_env;
      END;'
            );
     <!--define security-->

        DECLARE
        l_roles     OWA.VC_ARR;
        l_modules   OWA.VC_ARR;
        l_patterns  OWA.VC_ARR;
 
        ORDS.CREATE_ROLE(p_role_name  => 'ords-rest-access');
        l_roles(1)   := 'ords-rest-access';
        l_patterns(1):= '/api/v1/test_endpoint';
        ORDS.DEFINE_PRIVILEGE(
            p_privilege_name => 'test_privilege',
            p_roles          => l_roles,
            p_patterns       => l_patterns,
            p_modules        => l_modules,
            p_label          => '',
            p_description    => '',
            p_comments       => NULL);      
     
        COMMIT;
      
      END;

With this basic setup we test it and see if it works:

curl -u tomcat:tomcat http://localhost:8080/ords/test/api/v1/test_endpoint

First results

Wow, it works! We authenticate with Tomcat user and his role is passed to ORDS and checked against the one defined in the endpoint security constraint. That was easy!

Second try

Authenticating with Tomcat users is a nice training scenario, but what if we want to do something more based on a real production setup like a JNDI realm to do the same thing?  Just substitute the UserDatabaseRealm with JNDIRealm, provide necessary parameters and everything should be fine. So let's do just that. It would look something like this:

<Realm   className="org.apache.catalina.realm.JNDIRealm"
        connectionURL="ldap://localhost:389"
        userBase="ou=people,dc=mycompany,dc=com"
        userSearch="(mail={0})"
        userRoleName="memberOf"
        roleBase="ou=groups,dc=mycompany,dc=com"
        roleName="cn"
        roleSearch="(uniqueMember={0})"
/>

Doesn't seem too much different, does it?

Second results

Restart Catalina, send the request to the same endpoint, but this time authenticate with your LDAP user and...well, now you broke it. Just great.

Alright, let's see the logs up close. First, make sure that your logging.properties contains following lines to get more information about the authentication process:

org.apache.catalina.realm.level = ALL
org.apache.catalina.realm.useParentHandlers = true
org.apache.catalina.authenticator.level = ALL
org.apache.catalina.authenticator.useParentHandlers = true
oracle.dbtools.level=FINEST

We restart Catalina  again and we see very weird behaviour -- from inspecting the catalina.log it seems that Catalina authenticates our request, but then ORDS says that the request has not been authenticated. That is not a big issue for unsecured endpoints, but for endpoints that have been secured, like the one we defined earlier, it renders them inaccessible. The logs may look something like this:

10-Dec-2019 16:14:42.965 FINE [http-nio-8080-exec-9] org.apache.catalina.realm.CombinedRealm.authenticate Authenticated user [jgraniec] with realm [org.apache.catalina.realm.JNDIRealm]
10-Dec-2019 16:14:42.965 FINE [http-nio-8080-exec-9] org.apache.catalina.authenticator.AuthenticatorBase.register Authenticated 'jgraniec' with type 'NONE'
10-Dec-2019 16:14:42.965 FINE [http-nio-8080-exec-9] org.apache.catalina.authenticator.AuthenticatorBase.register Authenticated 'none' with type 'null'
10-Dec-2019 16:14:42.968 FINE [http-nio-8080-exec-9] . did not authenticate request

Hmm... that looks weird. Why Tomcat authenticates the request, but ORDS doesn't see it? If we look even deeper into the tutorials, we find this nice tutorial to authentication using JDBC realm(Tim Hall again to the rescue!): https://oracle-base.com/articles/misc/oracle-rest-data-services-ords-ba…

But there is only one significant difference that actually does the trick. It seems that setting up security constraint makes the whole thing work as intended. In the "$CATALINA_BASE/conf/web.xml" we should find something like this: 

<security-constraint>
        <web-resource-collection>
            <web-resource-name>ords</web-resource-name>
                <url-pattern>/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>*</role-name>
        </auth-constraint>
    </security-constraint>
    <login-config>
        <auth-method>BASIC</auth-method>
    </login-config>
    <security-role>
        <role-name>*</role-name>
    </security-role>
</security-coonstraint>

And even we see a note:

"Thanks to Marcel Boermann for helping me with this. I spent a lot of time trying to get this to work, then Marcel sent me an example "web.xml" file, which made it clear you need the full basic authentication setup in Tomcat, in addition to the normal ORDS security setup. (...)"

However, it doesn't seem right -- this method protects everything below "/*". If we want a public endpoint, available without any authentication, we would have to exclude it in our web.xml's protection pattern, which seems like the wrong idea and forces us to manage the privileges from two locations - the web.xml of Tomcat and privilege constraints in ORDS itself. What places this situation even further away from perfect is the fact that in WebLogic installation such a problem does not exist. Surely there must be something that can be done to help this behaviour.

The problem

After further inspection (and some decompilation), the culprit seems to be found. When the no security-constraint is present in Tomcat, the authentication is handled first by ORDS itself, using CatalinaAuthenticator (oracle.dbtools.auth.container.catalina.CatalinaAuthenticator to be exact). There we see something along the lines of:

 ...
request.login(username, new String(credential));
try {
    Principal principal = request.getUserPrincipal();
    if (!(principal instanceof User)) {
        return AuthenticationResult.unknown();
    }
    ...

Which doesn't look too bad at first, but if we take a closer look at it, we see that the expected instance of principal is of type "org.apache.catalina.User". In its JavaDoc we can read:

Abstract representation of a user in a UserDatabase.  Each user is optionally associated with a set of Groups through which he or she inherits additional security roles, and is optionally assigned a set of specific Roles.

So we see that this User class is intended to, essentially, be used only with UserDatabaseRealm. As can be seen from the code above, if the UserPrincipal is not an instance of User, the AuthenticationResults is unknown (failed in the result). We see that the request.login, in the end, calls Realm's authenticate method. The difference between UserDatabaseRealm and JNDIRealm is very slight. If correctly authenticated, UserDatabaseRealm returns new GenericPrincipal(String name, String password, List roles, Principal userPrincipal) while JNDIRealm returns new GenericPrincipal(String name, String password, List roles). This causes for the GenericPrincipal, when asked for a UserPrincipal  to return an instance of GenericPrincipal in case of JNDIRealm:

public Principal getUserPrincipal() {
    return (Principal)(this.userPrincipal != null ? this.userPrincipal : this);
}

The only problem is that GenericPrincipal is not in any correlation with User. They both implement java.security.Principal in the end sure, but GenericPrincipal is not castable to org.apache.catalina.User.

In order to work around this problem, we'll create a Valve that will insert our special UserPrincipal in the GenericPrincipal returned.

public class OrdsBasicAuthValve extends BasicAuthenticator {

    @Override
    protected Principal doLogin(Request request, String username, String password) throws ServletException {
        Principal principal = super.doLogin(request, username, password);
        if (principal instanceof GenericPrincipal) {
            GenericPrincipal gp = (GenericPrincipal) principal;
            if (!(gp.getUserPrincipal() instanceof User)) {
                User userPrincipal = new UserPrincipal(gp.getName(), gp.getPassword(), gp.getRoles());
                principal = new GenericPrincipal(gp.getName(), gp.getPassword(), Arrays.asList(gp.getRoles()), userPrincipal);
            }
        }
        return principal;
    }
}

Here, the UserPrincipal is a simple implementation of org.apache.catalina.User interface. A sample how this can look can be found here.

We make our Tomcat instance use our Valve and voilà! Our ORDS authentication works through Tomcat as intended, without any need for security-constraints and exactly the same way as in WebLogic.

Sidenote

For ORDS < 18.1.1 there seems to be an issue, where all Tomcat authentication handled by ORDS (so without security-constraint defined directly in Tomcat) fails, even for UserDatabaseRealm.
This is related to the BUG:26881221 ("Fix regression preventing authentication of Tomcat based users") that was fixed in version 18.1.1 of ORDS.
My recommendation would be to secure the whole application using the security-constraint and create a fallback realm, which would always authenticate as a 'fallback' user.
However, resulting behaviour will be a little bit different in the case of unsecured endpoints when it comes to the REMOTE_USER CGI variable. What would happen normally is:

• user authenticated successfully -> REMOTE_USER=*user*
• not authenticated -> REMOTE_USER=*schema_name*

In the proposed solution:

•  user authenticated successfully -> REMOTE_USER=*user* (just like before)
•  not authenticated -> REMOTE_USER=*fallback_realm_user*

For building your own realm, a very useful reference I found was: https://dzone.com/articles/how-to-implement-a-new-realm-in-tomcat