Steps for migrating DataStage project from 7.5 to 8.5

The below steps explains the DataStage parallel Job migration,

 

1. Export project backup from 7.5
2. Import the project backup to 8.5
3. Start adding the Environment Variables in 8.5 same as 7.5
4. Configuration file set-up same like 7.5
5. Compile the Job/Sequences to get the missing parameter/Values/Env variables.
6. Make sure the Plugins (DB/other passive stage you used in the 7.5) are with correct version are supported in 8.5 while you import.

Note :
Make sure you have license to import/compile/execute Parallel Job/ Sequence in the DS New version.
Make sure the DS Client services are enabled and installed with Patch if required.
Make sure the ODBC & TNS entries are made to talk with the database from DS jobs.
Make sure if you have to mount the database to bring as local.
Apart from the above steps, this is also a right time to change the job/Sequence design(for some,if required all) as well while you migrate.
If you a Re-designing the Job it should take as below,
2days -for Simple Job

3days -for Medium Job
5days -for Complex Job

For A single job migration it should take 30 Minutes (considering the network and other factor), so for 100 jobs it will take 6 days approximately if you are putting effort for 8 hours a day.Calculate accordingly.

For Server Job migration, same as above procedure.One thing you have to eliminate is Configuration file setup.
Make sure you have license to import/compile/execute Server Job/Sequence in the DS New version.

Important IBM Infosphere DataStage Directory Paths

 

OS	Default Directory	Directory Type
Linux UNIX	/opt/IBM/WebSphere/AppServer	Application server Installation Directory
Windows	C:\IBM\WebSphere\AppServer	Application server Installation Directory
Linux UNIX	/opt/IBM/ InformationServer/ Server/Datasets /opt/IBM/ InformationServer/ Server/Scratch	Data file Storage Directory
Windows	C:\IBM\ InformationServer\ Server\Datasets  C:\IBM\ InformationServer\ Server\Scratch	Data file Storage Directory
AIX Linux UNIX	/opt/IBM/InformationServer/Server/branded_odbc/	DataDirect Technologies Connect ODBC drivers
AIX Linux UNIX	/opt/IBM/InformationServer/Server/DSEngine. Files available: dsenv odbc.ini	DataStage Engine
Windows	C:\IBM\InformationServer\Server\DSEngine.	DataStage Engine
AIX Linux UNIX	/opt/IBM/InformationServer/Server/Projects/ Files available uvodbc.config	DataStage Projects
AIX Linux UNIX	/home/user_name	Home Directory
Windows	c:\Documents and Settings\user_name or c:\Users\user_name	Home Directory
Solaris	/ (root) /etc /etc/services /etc/inittab /etc/rc /tmp /var /var/tmp /usr/local/bin	Home Directory
Linux UNIX	/opt/IBM/InformationServer	Information Server installation Directory
Windows	C:\IBM\InformationServer	Information Server installation Directory
Windows	C:\IBM\InformationServer\client	InfoSphere Information Server client
AIX Linux UNIX	/opt/IBM/InformationServer/logs	Log Directory
Windows	c:\IBM\InformationServer\logs	Log Directory
AIX Linux UNIX	/opt/IBM/InformationServer/Server/PXEngine/ /tmp /ishome/Server/Scratch /ishome/Server/Datasets	Parallel Engine
Windows	C:\IBM\InformationServer\Server\PXEngine X:\tmp X:\IBM\InformationServer\Server\Scratch X:\IBM\InformationServer\Server\Datasets	Parallel Engine
Linux UNIX	/opt/IBM/ InformationServer/Server/ Projects	Project Directory
Windows	C:\IBM\InformationServer\ Server\Projects	Project Directory
AIX Linux UNIX	/tmp	Temporary Directory
Windows	x:\Documents and Settings\user_name\Local Settings\temp	Temporary Directory
Solaris	/tmp /var/tmp	Temporary Directory
Linux UNIX	/opt/IBM/InformationServer/_uninstall/installUpdates	Update Installer
Windows	C:\IBM\InformationServer\_uninstall\InstallUpdates.exe	Update Installer
Linux UNIX	/opt/IBM/WebSphere/AppServer64/bin/versionInfo.sh /opt/IBM/WebSphere/AppServer64/java/bin/java -fullversion	WebSphere Application Server installation
Windows	C:\IBM\WebSphere\AppServer\bin\versionInfo.bat C:\IBM\WebSphere\AppServer\bin\java -fullversion	WebSphere Application Server installation
Linux Unix	installdir/ASBServer/conf installdir/ASBNode/conf The typical installdir is /opt/IBM/InformationServer Create an empty text file called https.conf. In the above locations If you did not select HTTPS during the installation of these tiers	ASBServer Directory
Windows	installdir\ASBServer\conf installdir\ASBNode\conf The typical installdir is C:\IBM\InformationServer. Create an empty text file called https.conf. In the above locations If you did not select HTTPS during the installation of these tiers	ASBServer Directory

Understanding Tiers and Components Of IBM Infosphere Information Server 8.5 [ServiceTier and Metadata Tier] (Continued…)

Services tier

The services tier consists of the application server, common services for the suite, and product module-specific services and the computer where those components are installed.

Services for IBM InfoSphere Information Server
Some services are common to all product modules. Other services are specific to the product modules that you install. The services tier must have access to the metadata repository tier and the engine tier.

The common services include

Ø  Scheduling services

Ø  Logging services

Ø  Directory services

Ø  Security services

Ø  Reporting services

Ø  Core services

Ø  Metadata services

 

Metadata repository tier

The metadata repository tier consists of the metadata repository database and, if installed, the IBM® InfoSphere® Information Analyzer database and the computer where these components are installed.

The metadata repository tier includes the metadata repository database for IBM InfoSphere Information Server. The database is a shared component that stores design-time, runtime, glossary, and other metadata for product modules in the InfoSphere Information Server suite.

If InfoSphere Information Analyzer is installed, the metadata repository also includes one or more analysis databases. The analysis databases are used by InfoSphere Information Analyzer when it runs analysis jobs.

The services tier must have access to the metadata repository tier. When product modules store or retrieve metadata, services on the services tier connect to the metadata repository tier and manage the interaction between the databases and the product modules.

For InfoSphere Information Analyzer, the engine tier and the client tier must also have direct access to the analysis databases.

Tier relationships

 
The below diagram explains about the connection and relationship between the Various tiers. The tiers provide services like job execution, and metadata and other data storage for the product modules that you chosen to install.

Ø  Relationships differ depending on which product modules you install.

Ø  Client programs on the client tier communicate primarily with the services tier. However the IBM® InfoSphere® DataStage® and QualityStage™ clients will also communicate with the engine tier.

Ø  Various services within the services tier communicate with agents on the engine tier.

Ø  Metadata services on the services tier communicate with the metadata repository tier.

Ø  ODBC drivers on the engine tier communicate with external databases.

Ø  IBM InfoSphere Metabrokers and bridges on the client tier can import data from external sources.

       Some InfoSphere Metabrokers and bridges can export data. With the IBM InfoSphere Information Analyzer product module, the engine tier communicates directly with the analysis databases on the metadata repository tier. The InfoSphere Information Analyzer client also communicates directly with the analysis databases.

Understanding Tiers and Components Of IBM Infosphere Information Server 8.5 [Client Tier and Engine Tier] (Continued…)

Client tier

The client tier consists of the client programs and consoles that are used for development and administration.

The following tools are installed as part of the client tier, based on the products and components that you select

Ø  IBM InfoSphere® Information Server console

Ø  IBM InfoSphere DataStage® and QualityStage™ Administrator client

Ø  IBM InfoSphere DataStage and QualityStage Designer client

Ø  IBM InfoSphere DataStage and QualityStage Director client

Ø  IBM InfoSphere FastTrack client

Ø  IBM MetaBroker®s and bridges

Ø  IBM InfoSphere Connector Migration Tool

Ø  IBM InfoSphere Information Server istool command line. The istool framework is installed on the engine tier and client tier. Commands for IBM InfoSphere Information Analyzer, IBM InfoSphere Business Glossary, and InfoSphere FastTrack are installed on the clients only when those products are installed.

Engine tier

The engine tier consists of the logical group of engine components (the IBM® InfoSphere® Information Server engine components, service agents, and so on) and the computer where those components are installed.

Several product modules require the engine tier for certain operations. You install the engine tier components as part of the installation process for these product modules. The following product modules require the engine tier:

Ø  IBM InfoSphere DataStage®

Ø  IBM InfoSphere Information Analyzer

Ø  IBM InfoSphere Information Services Director

Ø  IBM InfoSphere Metadata Workbench

Ø  IBM InfoSphere QualityStage™

Ø  IBM InfoSphere Information Server istool command line. The istool framework is installed on the engine tier and client tier.

Commands for InfoSphere Information Analyzer and InfoSphere Metadata Workbench are installed on the engine tier only when those products are installed.

Note: Depending on the Operating systems (Windows/Linux&unix) some components will be available for Installation.

Understanding Tiers and Components Of IBM Infosphere Information Server 8.5

A Tier is a logical group of components within InfoSphere Information Server and the computers on which those components are installed.
 
The tiers provide services, job execution, and metadata and other data storage for your product modules.
 

Client: The client programs and consoles that are used for development and administration and the computers where they are installed.

Engine  :  The logical group of components (the InfoSphere Information Server engine components, service agents, and so on) and the computer where those components are installed. The engine runs jobs and other tasks for product modules

Services: Application server, common services, and product services for the suite and product modules and the computer where those components are installed. The services tier provides common services (such as metadata and logging) and services that are specific to certain product modules

Metadata Repository:  The metadata repository and, if installed, the IBM InfoSphere Information Analyzer analysis database and the computer where these components are installed. The metadata repository database contains the shared metadata, data, and configuration information for InfoSphere

Ownership History of DataStage

VMark acquired its main competitor in the PICK-on-UNIX market, UniData, to form Ardent Software.

Ardent acquired Dovetail Software for the core metadata technology that became MetaStage.

Informix acquired Ardent Software on March 2000 for a share swap worth $1.1 billion.

In 2001 Informix sold the database division to IBM. The remaining company was renamed Ascential Software.

Ascential acquired Torrent Systems for the parallel engine.

In 2005 IBM acquired Ascential Software and moved the products into the WebSphere Information Integration suite.

The October 2006 release of DataStage has integrated it into the new IBM Information Management Software brand.