Meta Integration® Model Bridge (MIMB)
"Metadata Integration" Solution

MIMB Bridge Documentation

MIMB Import Bridge from File System (CSV, Excel, XML, JSON, Avro, Parquet)

Bridge Specifications

Vendor ISO
Tool Name File System
Tool Version N/A
Tool Web Site https://en.wikipedia.org/wiki/File_system
Supported Methodology [Flat File] Multi-Model, Data Store (Physical Data Model) via Java API on CSV, TXT, AVRO, PARQUET, JSON File

Import tool: ISO File System N/A (https://en.wikipedia.org/wiki/File_system)
Import interface: [Flat File] Multi-Model, Data Store (Physical Data Model) via Java API on CSV, TXT, AVRO, PARQUET, JSON File from File System (CSV, Excel, XML, JSON, Avro, Parquet)
Import bridge: 'FileSystem' 10.0.0

IMPORTING FROM FILESYSTEM

This bridge is loading metadata from FileSystem files.
Please refer to the individual parameter's tool tips for more detailed examples.


Bridge Parameters

Parameter Name Description Type Values Default Scope
Root directory Enter the directory containing metadata files or specify it using browsing tool. Bridge provides up to 3 level browsing depth. DIRECTORY     Mandatory
Subdirectory filter Case sensitive filter of subdirectories expressed using the Java regular expression pattern syntax (e.g. '.*_xyz' - import subdirectories that end with _xyz).
You can define multiple conditions separated with spaces (e.g. 'x y' - import sub directories that called x or y). When a condition has spaces it must be quoted (e.g. 'x y' - import sub directories that called 'x y'). You can define an exclusion condition by putting the dash character in front of it (e.g. '-test' - skip all 'test' directories).
The Java regular expression documentation is available at: https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html
STRING      
File filter Similar to the directory filter parameter STRING      
Partition directories Files-based partition directories' paths.
The bridge tries to detect partitions automatically. It can take a long time when partitions have a lot of files.
You can shortcut the detection process for a partition by specifying it in this parameter.
Specify the partition directory path relative to the Root directory.
Use . to specify the root directory as the partitioned directory.

Separate multiple paths with the space character.
For example: dir1/dir2 dir3/dir4 dir5
STRING      
Sample size Enter the directory containing metadata files or specify it using browsing tool. Bridge provides up to 3 level browsing depth. NUMERIC      
Miscellaneous Specify miscellaneous options identified with a -letter and value.

For example, -m 4G -f 100 -j -Dname=value -Xms1G

-m the maximum Java memory size whole number (e.g. -m 4G or -m 2500M ).
-s import system objects that are skipped by default (e.g. -s)
-v set environment variable(s) (e.g. -v var1=value -v var2="value with spaces").
-j the last option that is followed by Java command line options (e.g. -j -Dname=value -Xms1G).
-hadoop key1=val1;key2=val2 to manualy set hadoop configuration options
STRING      
Incremental import Specifies whether to import only the changes made in the source or to re-import everything (as specified in other parameters).

True - import only the changes made in the source.
False - import everything (as specified in other parameters).

An internal cache is maintained for each metadata source, which contains previously imported models. If this is the first import or if the internal cache has been deleted or corrupted, the bridge will behave as if this parameter is set to 'False'.
BOOLEAN
False
True
True  

 

Bridge Mapping

Mapping information is not available

Last updated on Fri, 6 Apr 2018 18:57:59

Copyright © Meta Integration Technology, Inc. 1997-2018 All Rights Reserved.

Meta Integration® is a registered trademark of Meta Integration Technology, Inc.
All other trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.