Meta Integration® Metadata Management (MIMM)
"Metadata Management" Solution

README for Release Notes, Installation & Setup

Table of Contents

1. Overview

The Meta Integration® Metadata Management (MIMM) Application Server is
based on the Meta Integration® Repository (MIR) for metadata storage (in a database server),
and the Meta Integration® Model Bridge (MIMB) middleware for metadata harvesting.

2. Copyright Notice

3. Release Changes

v10.0 (2018/Q1 - WARNING: Actual release date and features are subject to major changes)

v9.1.0 (06/30/2017)

v9.0.2 (6/10/2016)

v9.0.1 GA (12/15/2015)

v8.0.3 GA (05/19/2015)
v8.0.2 LA (03/31/2015)

v8.0.1 (12/02/2014)

v8.0.0 (LA 10/01/2014)

v7.2 (11/01/2013)

v7.1 LA (04/05/2013)

v7.0.4 LA (10/12/2012)

v7.0.2 GA (1/31/2012)

General Availability for complete Metadata Management

v7.0.1 LA (10/6/2011)

Limited Availability and Limited Functionality for data modeling tool management

v6.2 (10/15/2010)

v6.0.6 (12/04/2009)

v6.0.5 (MIMB GA, MIMM GA) (09/28/2009)

v6.0.4 (MIMB GA, MIMM beta5) (06/02/2009)

v6.0.3 (MIMB GA, MIMM beta4) (01/28/2009)

v6.0.2 (MIMB GA, MIMM beta3) (10/31/2008)

v6.0.1 (MIMB GA, MIMM beta2) (07/22/2008)

v6.0.0 (MIMB GA, MIMM Beta1) (05/31/2008)

4. System requirements

4.1 Important preliminary disclaimer notice on all requirements

The following requirements only define the minimal requirements to run the application server with reasonable performance based on the provided tutorial, or small business use cases. The actual requirements for enterprise wide use cases based on larger models and configurations do require significantly greater resources to obtain acceptable performance.

The following requirements are based on:

Any other hardware/software configurations are acceptable as long as they provide the same (or better) results on the provided performance benchmark. In such case, if any problem is discovered (e.g. scalability or performance issues), then the customer must be able to reproduce the issue using an environment that conforms to the minimum performance requirements as defined herein.

Potential known issues include (but are not limited to) the following:

4.2 Web Client requirements

Users only need an internet browser:

4.3 Application Server Requirements

Hardware Minimum Requirements (based on physical hardware performance, not a virtual environment):

Operating System Requirements:

Application Server Engine Requirements:

Java Runtime Environment (JRE):

4.4 Database Server Requirements

Hardware Minimum Requirements (based on physical hardware performance, not a virtual environment):

For small deployments (or quick proof of concepts), the MIMM software package bundles PostgreSQL (for Windows only) as the MIMM Database Server (which can run on the same machine as the MIMM Application Server) See Application Server Installation and Configuration setup for details.

However, ONLY Oracle 12c is supported for large scale deployments where indexes must be kept in memory in order to avoid any disk I/O, and therefore obtain fast search response time. In such case, the Oracle 12c Server must have a enough memory (for KEEP buffer pool), a fast enough processor speed (for a single search), and enough cores (for multiple user searches in parallel). See Database on Oracle setup for details.

The MIMM Database Server can reuse your existing Oracle, SQL Server, or PostgreSQL server:

Database Administrator privileges are required to install/setup/uninstall the database.

In general, one must ALWAYS install the latest service packs for a given database version BEFORE creating the MIMM database. E.g., for Oracle 11.2 one is required to apply the patches to upgrade to 11.2.0.3, or whatever is the latest patch level at the time. In addition, Oracle 11.2.0.4 must have patch 17501296 applied.

Virtual Memory: For a Windows based database server, be sure to either:

Thus, you must have more than that much free disk space (at least 3 time the amount of memory or RAM) on the drive where the page file is defined to reside.

5. Metadata Management (MIMM) Database Server Setup

The MIMM Application Server requires the connection to an existing Database server for metadata storage (metadata repository)

However, a quick install for tests or QA purpose can be achieved by using the bundle PostgreSQL database.
See the section Metadata Management (MIMM) Application Server Setup for more details.

The following database setup scripts and instructions assume the following by default:
   Database Name: MM
   Database User: MM
   Database Password: = MM123!
The database name and user name can be changed, and the password should of course be different.

After the product is fully installed and web connectivity has been made, one may connect to a different database by way of the web based user interface at Tools -> Administration -> Database.

5.1 Database on Oracle

Create a user MM and a database MM with the following privileges:
sqlplus.exe SYS@<DB-NAME> as SYSDBA
 
   -- Delete previous user and database if needed
   -- DROP USER MM CASCADE;
 
   CREATE USER MM IDENTIFIED BY MM123!;
 
   GRANT CONNECT TO MM;
   GRANT CTXAPP TO MM;
 
   GRANT CREATE TABLE TO MM;
   GRANT CREATE VIEW TO MM;
   GRANT CREATE SEQUENCE TO MM;
   GRANT CREATE TRIGGER TO MM;
   GRANT CREATE PROCEDURE TO MM;
   GRANT CREATE TYPE TO MM;
 
   GRANT EXECUTE ON CTXSYS.CTX_DDL TO MM;
   GRANT EXECUTE ON DBMS_LOB TO MM;
   GRANT EXECUTE ON SYS.DBMS_LOCK TO MM;
 
   -- If you get the error "Database exception occurred: ORA-01950: no privileges on tablespace 'USERS'"
   -- ALTER USER MM QUOTA UNLIMITED ON USERS;

The Oracle 12 DB Administrator MUST optimize the KEEP buffer pool in order to avoid any disk I/O, and therefore obtain fast search response time. For more details and instructions, please refer to:
   %MIMM_HOME%\tomcat\conf\localhost\MM.xml

5.2 Database on Microsoft SQL Server

5.2.1 Database Requirement 1 - Full-text Search

SQL Server must have the Full-text Search component installed and running. This can be confirmed by making sure a service called "SQL Full-text Filter Daemon Launcher" (SQL Server 2008) is running in the Services panel or the SQL Server Configuration Manager.

This Full-text Search component can be added to any existing SQL Server, except for SQL Server Express. In case of SQL Server Express, the Full-text search component is only available in the "Advanced Services" package:
For 2008 R2, download the "SQLEXPRADV_xxx_xxx.EXE" file.
For 2012, download the "ENU\x64\SQLEXPRADV_x64_ENU.exe" file.

Make sure "Full-Text search indexing" check box is enabled for the MM database. This can be verified or changed by using the SQL Server Management Studio: first sign in, then right click on the MM database and select properties, finally go in the File area to find the above check box, and restart SQL Server.

5.2.2 Database Requirement 2 - Mixed-Authentication Mode

The Mixed-Authentication Mode is usually set during the SQL Server installation process.

The Mixed-Authentication Mode can be verified or changed by using the SQL Server Management Studio: first sign in, then right click on the root of the tree (instance of SQL Server Express), go to Security, and finally select "SQL Server and Windows Authentication mode"

5.2.3 Database Requirement 3 - TCP/IP Protocol Enabled

The TCP/IP Protocol must be enabled in the SQL Server Configuration Manager for both the named instance and the client protocols (Make sure you restart the service after changing).

5.2.4 Database Preparation

Login to SQL server as a user with server admin role and execute the following commands to create a database "MM" and a user "MM" with password "MM123!" (or another one):
   EXEC sp_configure 'clr enabled', 1
 
   RECONFIGURE
   Go
 
   CREATE LOGIN MM WITH PASSWORD = 'MM123!';
   CREATE DATABASE MM;
   ALTER DATABASE MM SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
   ALTER DATABASE MM SET READ_COMMITTED_SNAPSHOT ON;
   ALTER DATABASE MM SET MULTI_USER WITH ROLLBACK IMMEDIATE;
   ALTER AUTHORIZATION ON DATABASE::MM to MM;

5.2.5 Database Connection

Advanced SQL Server Administrators may define ("hard-code") a set of TCP/IP ports for SQL Server to run over the network. However, Microsoft now recommends to run the "SQL Server Browser" service which can be done either in the Services panel or the SQL Server Configuration Manager. For more information, read: How to: Configure Express to accept remote connections

The connection string syntax is:
   jdbc:sqlserver://<dbServer>:<dbPortNumber>;databasename=<dbName>

To connect to a named SQL server instance other than the default:

Note 1: The default database instance name for SQL Server Express is "sqlexpress, and "sqlserver" for any other SQL Server edition.
Note 2: The default SQL Server TCP/IP port number is 1433.

5.3 Database on PostgreSQL

Login to an existing database as a database superuser or a user who has CREATEROLE and CREATEDB privileges
psql.exe -h <HOST-NAME> -W -U <USER_NAME> -p <PORT> -d <DATABASE_NAME>
 
   -- Delete previous user if needed
   -- DROP USER "MM";
 
   -- If the user cannot be dropped due to any ownership issues, you'll need to reassign those objects to another user
   -- REASSIGN OWNED BY "MM" TO <OTHER-USER-NAME>;
   -- Or drop those objects
   -- DROP OWNED BY "MM"
 
   -- Create a user MM with LOGIN privilege
   CREATE USER "MM" LOGIN PASSWORD 'MM123!';
 
   -- Create a database MM with UTF8 encoding. You may use a different tablespace
   CREATE DATABASE "MM" WITH OWNER "MM" ENCODING 'UTF8' TABLESPACE pg_default;
 
   -- Connect to the database and add an extension
   \c "MM";
   CREATE EXTENSION intarray;

6. Metadata Management (MIMM) Application Server Setup

6.1 Server Installation and Configuration

The MIMM Application Server is installed as follows:

If your are using an existing database and do not wish to customize the application server (e.g. memory allocation, Windows services), then you can skip this step and go directly to the section on Application Server Execution and Initialization

Otherwise, go to the software home directory and "run As Administrator" the Setup utility (.bat on Windows or .sh on Linux). This setup utility will allow you to setup the configuration parameters defined below through a user friendly application. After any change on any panel (tab) below, remember to press the Configure button in order to perform the configuration changes. A dialog box will be issued to confirm success or failure (with error messages). Alternatively, this setup utility also works at the Windows command line or Linux shell, use the -help the options.

6.2 Application Server Upgrade

6.2.1 Understanding the Data Locations
Most application data is obviously located and your database server, you are responsible for regular backup of such database. Upgrading your application will also upgrade the associated database content (database schema, stored procedures, indexes and of course data). Therefore make sure you always backup your database before any upgrade.

Furthermore, the upgrade process may take several hours (on large repositories) and also need extra space for temp data during the migration. Therefore make sure the database has at least 20% free space.

Finally, it is also important to understand that the software installation directory (known as %MIMM_HOME% in this document) also contains some critical application data and application setup customizations that have to be taken into account in your backup or upgrade process, including:

6.2.2 Upgrade Process
We recommend the following upgrade process:

Finally, remember that any upgrade may also bring many new and improved import bridges that may harvest extra metadata. Therefore, any upgrade must be also followed by a re-harvest (import) of Models, and re-build of Configurations to take advantage of the new capabilities.

6.2.3 Version Specific Upgrade Issues and Recommendations
Upgrading to a new version may have version specific issues or recommendations that are listed at the bottom of the release notes: see Release Changes for more details.

6.3 Application Server Execution and Initialization

The easiest way to start the MIMM Application Server is to go to the software home directory and use the RestartApplicationServer utility (.bat on Windows or .sh on Linux).

The final initialization steps of the setup are performed over the web browser as follows:

  1. Connection
    Connecting to the server on Windows can be simply achieved by opening the Metadata Management link in the home directory. In all cases, you can connect to the server using your internet browser to open by default: http://localhost:19980/MM. Note that the default port of this URL number may have been changed by the Setup utility in the section Server Installation and Configuration..
  2. Database
    Define the connection to the previously created database (in the above steps), by providing the database type, user, password, and URL (JDBC connection). If you are using the PostgreSQL database bundled with the software package for Windows, then all these parameters should be already preset. Press Test Connection button to verify proper database connectivity. Finally, when the pressing the Save button, the MIMM Application Server will create all the necessary tables in the database.
  3. License
    Click on the Download License Information link to obtain the obtained your HostInfo.xml file that should be sent with your license request. Warning: Make sure your are NOT connected to any VPN during that step, then your license will work independently of your VPN connection. After you have received your MM.lic license file, browse for it and click on the Save License button.
  4. Login
    Login as "Administrator" with password "Administrator". Note that you should change that password later in the application by going to: Tools -> Administration -> Users)

6.4 Custom integration with authentication environments

MIMM is able to support three authentication methods:
  1. Native Authentication, where the password is managed by the software and stored within the database.
  2. LDAP Authentication, where the software does not manage or store the LDAP passwords at all. Instead, it is simply passed it through to LDAP in order to authenticate.
  3. External Authentication such as Single Sign On (SSO), where the software does not perform any authentication, and leaves that responsibility to a local single sign on service managed by the customer.

In Tools->Administration->Users one may specify either:

  1. Mixed Native and LDAP authentication where users may be authenticated either as native users or LDAP users
  2. External authentication where the system does not perform any authentication, leaving it up to a local Single Sign On environment.

6.4.1 Native Authentication Configuration Issues
There are no specific configuration steps for Native Authentication.
6.4.2 LDAP Authentication Configuration Issues
There are no special server configuration issues for LDAP Authentication. LDAP connectivity configuration is documented in the online help.
6.4.3 Windows Authentication Issues
It is also possible to enable the Application Server to obtains authentication for users from Windows authentication via the browser (client). This way, users will automatically be authenticated if they are running from a Windows session.

To do so, one must install a third party product named Waffle (Windows Authentication Functional Framework) as an addon (see here):

  1. Please ensure that all LDAP settings are correct and users are able to log into the product via LDAP authentication. LDAP connectivity configuration is documented in the online help.
  2. Unzip the Waffle zip.
  3. Copy all the jar files from it to %MIMM_HOME%\tomcat\lib
  4. Open %MIMM_HOME%\tomcat\conf\web.xml. Search for "Windows authentication support". Uncomment the block following that.
  5. Restart MIMM.
  6. You should have windows authentication enabled now. Any valid windows user will be logged in as guest by default as long as licensing allows it. If you need to get an administrator interface, you can access: http://host:port/Admin
  7. Provide connection information for the database you created above.

Note: Waffle is designed around Windows libraries and thus it is recommended that you use a Windows OS based machine as the Application Server. While it is possible to use Waffle on a Linux based machine, it will require a great deal of manual setup and compilation. Please follow the Waffle documentation for such an implementation (see here).

Note: When using Waffle on a Windows based Application Server (as is recommended) you must run as run the MM software as a Windows service (not as an Application) in order for Waffle to work properly.

Note: Automatic Windows authentication will not allow one to use the browser refresh (f5) with IE 8.x when used as the client browser. Refresh will force a re-authentication on IE 8.x browsers and will not be automatically authenticated. If this occurs, the user must close all instances of the browser and start again. To avoid this issue, one must use IE 9.x or later or another approved browser (see System requirements) In addition, for Internet Explorer and Firefox, you must configure the browser at each client to support automatic Windows authentication. Please refer to the Waffle web site here.

6.5 Custom integration for Secure Socket Layer (SSL) communication

Important Disclaimer: SSL is primarily used for HTTPS secure communications from the web browser clients to the MIMM Server itself. Such common HTTPS setup can be fully achieved with the Setup utility as explained in Server Installation and Configuration. The following steps are provided for illustration purpose only (manual steps), describing what the Setup utility already performs automatically. THEREFORE, YOU DO NOT HAVE TO PERFORM THESE STEPS BELOW.

If you want to manually install a your own certificate, you must:

  1. Change the referenced (in server.xml) connector entry parameters (keystoreFile and keystorePass) to point to the correct keystore file and password.
  2. Import that certificate into the JRE that is being used by this tomcat. The default JRE is located under:
       %MIMM_HOME%/jre.
  3. Use the following commands:
       cd %MIMM_HOME%/jre/lib/security
       move jssecacers jssecacers.old
       %MIMM_HOME%/jre/bin/keytool -importkeystore -srckeystore {your_keystore} -keystore jssecacerts
       %MIMM_HOME%/RestartApplicationServices.bat

After the configuration, use the default URL to Access MIMM: https://localhost:19980/MM

Or use the ports specified in the server.xml file. For example:
   <Connector port="19980" maxThreads="200"
              scheme="https" secure="true" SSLEnabled="true"
              keystoreFile="conf\keystore" keystorePass="changeit"
              clientAuth="false" sslProtocol="TLS" />

6.5.1 Configuring MIMM to securely connect via HTTPS to another MIMM server for Metadata Harvesting

Important Disclaimer: the following steps are needed ONLY IF you use a self signed certificate for SSL (WHICH IS NOT RECOMENDED), AND ONLY in the case of configuring MIMM to securely connect via HTTPS to another MIMM server for Metadata Harvesting. Only in such exceptional use case, then the following additional steps have to be performed

In order to support HTTPS from a MIMM Server acting as the "Metadata Manager" to a MIMM Server acting as "Metadata Harvesting" Agent, the Administrator needs to import the trusted certificate that the MIMM "Harvesting Agent" Server is using into the JRE that the MIMM "Metadata Manager" server is using. The following page describes the process: http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html.

The command looks like the following:
cd %MIMM_HOME%\jre\lib\security
..\..\bin\keytool.exe -import -alias john -file YourOwnCertificate.cer -keystore jssecacerts

6.5.2 Configuring MIMM to securely connect via LDAPS to the Enterprise Directory
In LDAP Authentication, the user password is not managed by the software and is simply passed through to the LDAP system.

Note: this password is not encrypted when communicated between the client and the server. Thus, in order to ensure encryption you may wish to specify HTTPS protocol communication, as above.

Note: this password is also not encrypted when communicated between the server and LDAP. Thus, in order to ensure encryption you may wish to also specify LDAPS protocol communication and thus use SSL to encrypt.

In order to support LDAPS, the MIMM Tomcat service does not itself need to be configured to work with LDAPS for encryption of passwords. However, to enable secure SSL communication between MIMM and LDAP servers the Administrator needs to import the trusted certificate that the LDAP server is using into the JRE that the MIMM Application server is using. The following page describes the process: http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html.

The command looks like the following:
cd %MIMM_HOME%\jre\lib\security
..\..\bin\keytool.exe -import -alias john -file YourOwnCertificate.cer -keystore jssecacerts

This is an entirely different certificate from the one used by the HTTPS protocol.

6.5.3 SSL Security Vulnerabilities

Poodle is a "Man In The Middle" (MITM) vulnerability which needs to be primarily fixed server side. An attacker can trick the server into downgrading the encryption protocol used to communicate. The servers should be configured to disallow TLS fallback, or to disable SSLv3 as a valid protocol.

If Tomcat has been configured with SSL support, the customer should add the following to the connector description in the %MIMM_HOME%\tomcat\conf\server.xml
   sslEnabledProtocols="TLSv1.2,TLSv1.1,TLSv1"

6.6 Security and Vulnerability Considerations

Important Disclaimer: Apache Tomcat is bundled within MIMM. For security and vulnerability reasons make sure you apply any current patches from Apache for the particular version of Tomcat.

For additional protection we recommend enabing in tomcat to check for allowed referrer by :

  1. Edit %MIMM_HOME%\tomcat\conf\web.xml
    1. Uncomment the two filter sections in the 'Checks referer is allowed' section. The variable ${server.fqdn} will be substituted with the value of M_SERVER_FQDN in tomcat.properties
    2. Add as many init-param sections as needed to allow reference from other URLs.
  2. Edit %MIMM_HOME%\tomcat\conf\tomcat.properties by:
    1. Changing the M_SERVER_FQDN variable from 'localhost' to '<myMMServer.myDomain>'

7. Model Bridge (MIMB) Metadata Harvesting Setup

The Metadata Integration or Metadata Harvesting from third party databases, data modeling, data integration or business intelligence tools is performed by the integrated Meta Integration® Model Bridge (MIMB) software. By default, the installer software deploys and configures both MIMM and MIMB on the same machine, where the MIMM Application Server accesses the MIMB Web Services locally. MIMB can also be installed and configured as a remote MIMB Agent on another machine. This is very useful in architecture deployments where the metadata management server is:

Essential customizations (e.g. directories, memory) of the MIMB Application Server can be performed in the following configuration file:
   %MIMM_HOME%\conf\conf.properties

Recommended customizations include:

When the MIMB Application Server is used a local metadata harvesting agent connected to a MIMM Application Server on the cloud, the additional customizations are needed in the following configuration file:
   %MIMM_HOME%\conf\agent.properties

8. User Interface Look & Feel Customization

8.1 Login and Headers

Customize the following files and directories using the embedded instructions (in comments):
   %MIMM_HOME%\conf\ressources\MM.properties
   %MIMM_HOME%\conf\ressources\web\

8.2 Metadata Explorer for Business Users

Customize the following files using the embedded instructions (in comments):
   %MIMM_HOME%\conf\ressources\MetadataExplorer.xml

9. REST API SDK

The REST API SDK documentation is available within the Metadata Manager UI by going to the Help menu (top right corner) under Help on REST API SDK, or go directly to: http://localhost:19980/MM/REST-API/.

For illustration purpose, a glossary definition search demo is also available at http://localhost:19980/MMGlossaryClient/. This demo assumes a server URL located at "http://localhost:19980", and a glossary in a configuration called "Published". REST API application developers are familiar with the above documentation, and can find the demo source code in Chrome > top right menu > More tools > Developer tools.