Home
News
Meta Integration June 2011 - Meta Integration releases
new version 7
Site Map
Home < width=46 height=34>

Meta Integration® Model Bridge (MIMB)
"Metadata Integration" Solution

< Supported Tools | Readme < Description | About Standards width=17 height=22>

Known Limitations as of 2011/12/01

Table of Contents

  1. Consistency Check of the Source Model
  2. Metadata Import/Export is limited to the "Bridge Specifications"
  3. Potential Loss of Metadata from Tool A to Tool B
  4. Limitations on Import/Export Round-Trip
  5. Limitations on Export Bridge's Merge/Update Capabilities
  6. API based Bridges
  7. Multi-file based Bridges
  8. Graphical Layout is carried only through some Bridges
  9. Parsing of Transformation Expressions in Import Bridges
  10. Conversions of Transformation Expressions in Export Bridges
  11. Internationalization Aspects
  12. Multi-Model Import Bridges for Incremental Metadata Harvesting
  13. Model Bridge Mapping Specifications

1 - Consistency Check of the Source Model

The user can select between "basic" or "detailed" consistency check of the imported model before the export to the target tool. This kind of model consistency check can detect disconnected relationships, foreign keys not connected to any primary or alternate key, etc. Having a centralized model validation between import and export allows to continuously improve the consistency check algorithms, and avoids each export bridge to retest for the same issues. Consistency checks can report "warnings" in the source model, or "errors" that are fatal and will prevent the export to any tool. The solution is to fix the original model, and then import it again. If the user purposely decides to turn off all consistency check (e.g. when the tool is not available to fix the source model), then the imported model may break the export bridge, or produce an invalid model that may break the target tool. MITI assumes no responsibility when consistency check is turned off.

2 - Metadata Import/Export is limited to the "Bridge Specifications"

The metadata mapping and transformations implemented by Meta Integration® Model Bridge (MIMB) are limited to the published model mapping specifications for each integrated tool. Such specifications are part of the MIMB online help, and are also published on the web at http://www.metaintegration.net/Products/MIMB/SupportedTools.html (select/click on a bridge to get the full specifications including all the bridge parameters/options and detailed mapping specifications).

3 - Potential Loss of Metadata from Tool A to Tool B

There are 7 points of potential metadata loss from Tool A to Tool B:

  1. Tool A's Methodology/Metamodel Support & Implementation,
  2. Tool A's Export Capabilities,
  3. MIMB's Import Bridge for Tool A,
  4. MIR's Methodology/Metamodel Support & Implementation,
  5. MIMB's Export Bridge for Tool B,
  6. Tool B's Import Capabilities,
  7. Tool B's Methodology/Metamodel Support & Implementation.

MITI provides the best possible metadata mapping across methodologies and tools. However, some modeling concepts may not be available in Tool A, MIR, or Tool B. In such a case, the import & export bridges try to use the descriptions or notes to carry such modeling concepts all the way from the source to the target tool without any loss.

4 - Limitations on Import/Export Round-Trip

There are an additional considerations when performing what is often referred to as a "round-trip", or importing from tool A, exporting to tool B, editing the model (or not) in tool B, re-import this model in from tool B, and exporting back to tool A. In this case, the 7 points of potential metadata loss identified above are compounded so that there are now 7 additional points of loss:

The fidelity of some simple types of metadata may be well maintained, where there is a direct, one-to-one, translation from tool A --> MIR's methodology --> tool B and then back again. Examples include tools with the same methodology and metadata coverage (e.g., a data modeling and design tool to another data modeling or design tool), or simple elements like the name, description, layout, etc. This process is referred to as "round-tip migration", and may work successfully given these restrictions.

However, it is never recommended that a combination of import-> export-> edit-> export-> import (i.e., "round-trip engineering") be expected to retain fidelity when there is a migration from tool A to tool B (and vice-versa) of methodology, metadata coverage or any of the known limitations identified above. What round-trip engineering implies is a forward-engineering and then reverse engineering process, which has never been a reality in systems engineering. One should not expect the forward engineering of simple to complex transformations (e.g., forward engineering from a spreadsheet to an ETL tool and then reverse engineering from the ETL tool back to a spreadsheet) would be reversible, or even bear much resemblance to the original.

The proper process is to treat the forward-engineering step as "irreversible", treat the reverse engineering step as a separate process, and not expect to effectively compare the original will the "round-trip" result, nor re-use the result for further engineering of the metadata.

Some examples:

Although each of the above steps can use the same Excel format (MetaMap), the roundtrip reengineering is not recommended and is likely to produce results which are not useful for all of the reasons identified above. Furthermore, like most destination tools, Informatica PowerCenter, as an example, is unable to consume the exported metadata correctly, i.e., it is unable to compare/merge/integrate different versions of complex transformations. (Please see the section on Limitations in the Receiving Environment for Merge/Update of Metadata.)

5 - Limitations on Export Bridge's Merge/Update Capabilities

The export bridges generally depend upon the tool receiving the metadata to integrate the metadata correctly, i.e., provide compare, merge and integrate capabilities. Such capabilities are:

In order to accommodate these limitations in the destination tool, the following MIMB export bridges offer "limited" metadata update capabilities (Please see the bridge parameter tooltips for more details):

6 - API based Bridges

Some MIMB bridges rely on the tool's API to import/export metadata, for example:

Such MIMB bridges require these tools to be installed with their API properly setup on the machine (PC) where the MIMB bridge is executed.

7 - Multi-file based Bridges

When multiple files are involved, all must be accessible to the MIMB bridge.

Some MIMB bridges have parameters of type "Directory" instead of just "File", and therefore require access to a directory of multiple files, for example:

Some MIMB bridges have parameters of type "File" which support the notion of "include" files, and therefore require access to all the specified include file paths, for example:

8 - Graphical Layout is carried only through some Bridges

Some MIMB bridges convert the graphical information between tools, including conversion of the model layout between various notations like IDEF1X data modeling, and UML object modeling. The following MIMB bridges carry graphical information:

However, some MIMB bridges do not transfer the graphical information of the model layout. The primary reason for this limitation is that the import / export capabilities of most tools do not provide graphical information. In other words, their published file formats and/or Application Programming Interface (API) cover the semantic (each of the modeling concepts), but not the graphical information (i.e. the concept's associated shape sizes and positions). Furthermore, when available, such graphical information is not easily reusable in the target tool. This problem is also true for tools sharing the same methodology (e.g. a tool may allow a graphical layout that is not graphically implementable in another equivalent tool). This problem is accentuated when crossing methodology boundaries (e.g. IDEF1X to UML).

9 - Parsing of Transformation Expressions in Import Bridges

9.1 - SQL Expression Analyzer Based Import Bridges

Some MIMB Import Bridges depend on "Transformation Expression Parsing" to capture the transformation semantic (i.e. ClassifierMap and FeatureMap). Such parsing might be limited by the complexity of the transformation expression (e.g. proprietary SQL language extensions). For expressions which can be parsed successfully, the lineage dependencies are captured. Expressions too complex to parse are captured only in textual form, no lineage is derived from the expression.

MIMB bridges depending on "Transformation Expression Parsing" include the following import bridges:

9.2 - SQL Expression Analyzer intrinsic limitations

Supported features

9.2.1 - Limitations on SQL99 Keywords

The list below contains all the keywords currently supported by the expression analyzer. Note that some keywords are used for compound terms like "order by", "group by", "union join":

ALL, AND, ANY, AS, ASC, AUTO, BETWEEN, BOTH, BY, CASE, CAST, CHAR, CHARACTER, CONVERT, CREATE, CROSS, DATE, DEC, DECIMAL, DESC, DISTINCT, DOUBLE, DOUBLE_PRECISION, ELSE, END, EXCEPT, EXISTS, FLOAT, FOR, FROM, FULL, GROUP, HAVING, IF, IN, INNER, INT, INTEGER, INTERSECT, IS, JOIN, LEADING, LEFT, LIKE, LONGVARCHAR, MINUS, NOT, NULL, NUMBER, NUMERIC, ON, ONLY, OR, ORDER, OUT, OUTER, OVER, PARTITION, PREFILTER, PROCEDURE, READ, REPLACE, RIGHT, SELECT, SMALLINT, SUBPARTITION, THEN, TIME, TIMESTAMP, TOP, TRAILING, UNION, USING, VARCHAR, VIEW, WHEN, WHERE, WITH, WRITE

Any word which is not part of the list above (unless handled specially by a tool specific lexer) will be treated as part of name for an identifier or a function call.

List of unsupported SQL99 keywords. Many of these keywords are not applicable to select statements but this is the full list for completeness:

ABSOLUTE, ACTION, ADD, ADMIN, AFTER, AGGREGATE, ALIAS, ALLOCATE, ALTER, ARE, ARRAY, ASSERTION, AT, AUTHORIZATION, BEFORE, BEGIN, BINARY, BIT, BLOB, BOOLEAN, BREADTH, CALL, CASCADE, CASCADED, CATALOG, CHECK, CLASS, CLOB, CLOSE, COLLATE, COLLATION, COLUMN, COMMIT, COMPLETION, CONNECT, CONNECTION, CONSTRAINT, CONSTRAINTS, CONSTRUCTOR, CONTINUE, CORRESPONDING, CUBE, CURRENT, CURRENT_DATE, CURRENT_PATH, CURRENT_ROLE, CURRENT_TIME, CURRENT_TIMESTAMP, CURRENT_USER, CURSOR, CYCLE, DATA, DAY, DEALLOCATE, DECLARE, DEFAULT, DEFERRABLE, DEFERRED, DELETE, DEPTH, DEREF, DESCRIBE, DESCRIPTOR, DESTROY, DESTRUCTOR, DETERMINISTIC, DICTIONARY, DIAGNOSTICS, DISCONNECT, DOMAIN, DROP, DYNAMIC, EACH, END-EXEC, EQUALS, ESCAPE, EVERY, EXCEPTION, EXEC, EXECUTE, EXTERNAL, FALSE, FETCH, FIRST, FOREIGN, FOUND, FREE, FUNCTION, GENERAL, GET, GLOBAL, GO, GOTO, GRANT, GROUPING, HOST, HOUR, IDENTITY, IGNORE, IMMEDIATE, INDICATOR, INITIALIZE, INITIALLY, INOUT, INPUT, INSERT, INTERVAL, INTO, ISOLATION, ITERATE, KEY, LANGUAGE, LARGE, LAST, LATERAL, LESS, LEVEL, LIMIT, LOCAL, LOCALTIME, LOCALTIMESTAMP, LOCATOR, MAP, MATCH, MINUTE, MODIFIES, MODIFY, MODULE, MONTH, NAMES, NATIONAL, NATURAL, NCHAR, NCLOB, NEW, NEXT, NO, NONE, OBJECT, OF, OFF, OLD, OPEN, OPERATION, OPTION, ORDINALITY, OUTPUT, PAD, PARAMETER, PARAMETERS, PARTIAL, PATH, POSTFIX, PRECISION, PREFIX, PREORDER, PREPARE, PRESERVE, PRIMARY, PRIOR, PRIVILEGES, PUBLIC, READS, REAL, RECURSIVE, REF, REFERENCES, REFERENCING, RELATIVE, RESTRICT, RESULT, RETURN, RETURNS, REVOKE, ROLE, ROLLBACK, ROLLUP, ROUTINE, ROW, ROWS, SAVEPOINT, SCHEMA, SCROLL, SCOPE, SEARCH, SECOND, SECTION, SEQUENCE, SESSION, SESSION_USER, SET, SETS, SIZE, SOME, SPACE, SPECIFIC, SPECIFICTYPE, SQL, SQLEXCEPTION, SQLSTATE, SQLWARNING, START, STATE, STATEMENT, STATIC, STRUCTURE, SYSTEM_USER, TABLE, TEMPORARY, TERMINATE, THAN, TIMEZONE_HOUR, TIMEZONE_MINUTE, TO, TRANSACTION, TRANSLATION, TREAT, TRIGGER, TRUE, UNDER, UNIQUE, UNKNOWN, UNNEST, UPDATE, USAGE, USER, VALUE, VALUES, VARIABLE, VARYING, WHENEVER, WITHOUT, WORK, YEAR, ZONE

9.2.2 - Limitations on Database extensions

In addition to the list of SQL99 keywords, the expression analyzer has been enhanced to support common database specific constructs, detailed in the list below.

SQL 99 extensions

Support for "with check option" which is a SQL 99 construct.
CREATE VIEW v AS SELECT * FROM customer WITH CHECK OPTION;

Oracle extensions

Support for Oracle older join syntax
SELECT a, b FROM t1, t2 WHERE t1.a (+)= t2.b;

Support for Oracle partitioning functions
SELECT month || '/' || year, product_name, amount, RANK() OVER(PARTITION BY month, year ORDER BY amount DESC) month_rank
FROM monthly_turnover
ORDER BY year ASC, month ASC, month_rank ASC;

SQL Server extensions

Support for SQL Server "top" functionality
SELECT TOP(3) first_name FROM customer;

Support for SQL Server XML extensions
SELECT [CatalogDescription].value(N'declare namespace p1="http://schemas.microsoft.com/sqlserver/2004/07/adventure-works/ProductModelDescription";
declare namespace html="http://www.w3.org/1999/xhtml";
(/p1:ProductDescription/p1:Summary/html:p)[1]', 'nvarchar(max)') AS [Summary]
FROM [SalesLT].[ProductModel]
WHERE [CatalogDescription] IS NOT NULL;

DB2 extensions

The CONCAT keyword in DB2 can be used as a function (supported) or as an operator (not supported)
SELECT CONCAT(first_name, last_name) FROM customer;
SELECT first_name CONCAT last_name FROM customer;

Teradata extensions

The Teradata specific aliasing construct is supported
SELECT last_name(NAMED name) FROM customer;

The Teradata specific data format is supported
SELECT birth_date AS DATE FORMAT 'mm/dd/yyyy' FROM customer;

The Teradata "lock row for access" is supported
LOCK ROW FOR ACCESS SELECT first_name, last_name FROM customer; LOCK ROW FOR ACCESS MODE SELECT first_name, last_name FROM customer;

9.2.2 - Limitations on Syntax

The WITH clause
The expression analyzer does not support the WITH clause for temporary tables.

WITH dept_costs AS (
SELECT department_name, SUM(salary) dept_totalFROM employees e, departments dWHERE e.department_id = d.department_id
GROUP BY department_name),avg_cost AS (
SELECT SUM(dept_total)/COUNT(*) avg
FROM dept_costs)
SELECT * FROM dept_costs
WHERE dept_total > (SELECT avg FROM avg_cost)
ORDER BY department_name;

The PIVOT clause
The expression analyzer does not support the PIVOT clause to pivot rows and columns.

SELECT 'AverageCost' AS Cost_Sorted_By_Production_Days, [0], [1], [2], [3], [4]
FROM (SELECT DaysToManufacture, StandardCost FROM Production.Product)
AS SourceTable
PIVOT (AVG(StandardCost) FOR DaysToManufacture IN ([0], [1], [2], [3], [4])) AS PivotTable;

Hierarchical query
The expression analyzer does not support hierarchical queries.

SELECT last_name, employee_id, manager_id, LEVEL
FROM employees
START WITH employee_id = 100
CONNECT BY PRIOR employee_id = manager_id;

9.2.4 - Limitations on Data Type Support

The expression analyzer only supports a subset of all data types supported by the databases, as can be seen in the list of supported keywords.

10 - Conversions of Transformation Expressions in Export Bridges

Some MIMB Export Bridges depend on "Transformation Expression Conversions" to restore the transformation semantic (ClassifierMap and FeatureMap). Such conversion might be limited by the complexity of the transformation expression. If the target tool accepts expressions in textual form, such expressions too complex to restore are converted only in textual form, as defined in the source tool language. If the expression can be successfully restored, it is transformed into the target tool language.

MIMB bridges depending on "Transformation Expression Conversions" include the following export bridges:

11 - Internationalization Aspects

The internationalization of the MIMB bridge names, parameters, tooltips and messages is managed by XML files within [MIMB-INSTALL-DIR]\conf\MIRI18n\

The support for transferring internationalized (e.g. multi-byte charater sets) through MIMB is achieved via the UTF8 encoding in the MIR neutral metamodel. The MIMB import and export bridges which cannot automatically detect the encodings of the tools, offer a parameter called "Encoding" allowing the user to specify one. There are however a few minor limitations:

12 - Multi-Model Import Bridges for Incremental Metadata Harvesting

Import bridges can be used for metadata harvesting into the repository of a Metadata Management (MM) solution. The metadata to be harvested can be very large, especially from Business Intelligence (BI) environments which may contain thousands of reports. Consequently some import bridges are implemented as a "Multi Model" bridge managing the large amount of metadata in multiple Model files. This approach also allows for incremental harvesting where the import bridge can check what has changed since the last import date, and return an updated "table of contents" indicating which models have been added, modified or deleted and import only the new or updated models. Such feature is supported only by a limited amount of import bridges listed below:

13 - Model Bridge Mapping Specifications

Each supported import and export bridge has a mapping specification defined for it, available at the Supported Tools page. This mapping specification is intended to define the scope of metadata captured (by an import bridge) or populated (by an export bridge) in the 3rd party tool or format (tool and metamodel naming). Therfore, it can only be used to analyze the scope of the metadata covered by the bridge algorithms. However, it is not intended to be a complete documentation of all the mapping algorithms involved in migrating metadata from any tool to any other tool, as the algorithms includes expression parsing, expression conversions and other complex operations which may not be expressed as a mapping specification.

< Supported Tools | Readme < Description | About Standards width=17 height=22>

Copyright © Meta Integration Technology, Inc. 1997-2012 All Rights Reserved.
Meta Integration® is a registered trademark of Meta Integration Technology, Inc.
All other trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.
http://www.metaintegration.com