|
Thursday, September 24, 2009
Free Online Training in IBM Cognos 8 PowerPlay
Monday, September 21, 2009
Migration Step by Step tasks
This is a tentative migration road map. All durations and sub tasks are estimates, more duration is required for Planning. All assumption are based on WINDOWS machines.
| Activity | Sub task | Duration | Res |
| Sandbox migration | |||
| Install series 7 | 1 day | 1 | |
| Instal Cognos8 | 1 day | 1 | |
| Configure LDAP/Access Manager | 1 day | 1 | |
| Import Sample Catalogs and Reports | 0.5 day | 1 | |
| Import Impromtu reports | 0.5 day | 1 | |
| Import Sample Cubes and reports | 0.5 day | 1 | |
| Migrate Catalog, IMPCAT2XML | 0.5 day | 1 | |
| Copy Unsecured cubes | 0.25 days | 1 | |
| Copy Secured Cubes | 0.25 days | 1 | |
| Publish a cube as a package | 0.25 days | 1 | |
| Create data source connection for catalog and cubes | 0.25 days | 1 | |
| Test them to see if they work | 1 day | 1 | |
| Use Migrate to C8 to migrate impromptu and PP reports | 1 day | 1 | |
| Use Deploment Manager to create a dmp file of web content such as newsboxes | 1 day | 1 | |
| Manually deploy reports to CC | 1 day | 1 | |
| Test your reports/open with AS/RS | 1 day | 1 | |
| 11 days | 1 | ||
| Actual Migration | 1 | ||
| Pre Clean the environment | 1 | ||
| Remove broken url's | 0.5 days | 1 | |
| Remove unwanted reports | 1 day | 1 | |
| Identify secured and unsecured cubes | 1 day | 1 | |
| Create administrator as the owner for all catalogs and cubes | 1 day | 1 | |
| Identify any image files | 0.5 days | 1 | |
| 4 days | 1 | ||
| Organize your source | 1 | ||
| Create a folder for Catalogs | 0.25 days | 1 | |
| Create a folder for Cubes | 0.25 days | 1 | |
| Create a folder for Impromptu reports | 0.25 days | 1 | |
| Use a deployment manager to create dmp files for web content | 0.5 days | 1 | |
| Create a folder for PPX reports | 0.25 days | 1 | |
| 1.5 days | 1 | ||
| step by step Migration | 1 | ||
| CATALOG Migration | 1 | ||
| Migrate Catalogs using Impcat2 XML | 1 day | ||
| Publish FM packages with Datasource connection | 1 day | 1 | |
| Test your migration extensively | 3 days | 1 | |
| 5 days | 1 | ||
| CUBE and MDL Migration | 1 | ||
| FTP your MDL and Cubes | 1 day | ||
| Edit the COGNOS.INI file for datasource connection | 0.5 day | 1 | |
| OPEN MDL in newer version of Transformer and save | 1 day | 1 | |
| Publish cubes using Transformer/FM | 2 days | 1 | |
| Create data source connections for cubes | 1 day | 1 | |
| Test your Cubes | 2 days | 1 | |
| 6.5 days | 1 | ||
| Migrate to C8 | 1 | ||
| Open CMD window | 0.25 ays | 1 | |
| Browse to the location | 0.25 days | 1 | |
| Type MigratetoC8 <source> <target> | 0.25 days | 1 | |
| Look at the target directories | 1 day | 1 | |
| Repeat the above 4 steps for various folders(IWR,IMR,PPX,DMP etc) | 1 day | 1 | |
| 2.75 days | 1 | ||
| DeployToC8 | 1 | ||
| Two Option-Manual or auto | |||
| Copy the .zip directory from target location to c8 deploment location | 0.5 days | 1 | |
| Check Migration service is on C8 ( defauly is always on) | 0.5 day | 1 | |
| Import the deployment file | 0.25 days | 1 | |
| Test your reports | 2 days | 1 | |
| AUTO(optionl) | 1 | ||
| Browse to location c8/deplotC8 | 0.25 days | 1 | |
| Type DeploytoC8 <source><target> | 1 day | 1 | |
| Test your reports | 2 days | 1 | |
| 5.75 days | 1 | ||
| Post Migration Tasks | 1 | ||
| Replace IQD's with package/report source | 7 days | 1 | |
| Disable/Interoperate series 7 and Cognos8 | 5 days | 1 | |
| Recreate broken reports | 10 days | 1 | |
| 22 days | 1 | ||
| Total duration (approx) | 60 days | 1 | |
Wednesday, August 12, 2009
Migration: Best Practice to Replace IQD's
The first challenge in the post migration phase is to gradually reduce dependence on Impromptu Query Definitions (IQD) as a data source in Transformer.
The reasons for moving away from IQD’s are:
1. IQD’s are files and hence should be avoided
2. You might need to maintain Impromptu licenses
3. If you have created IQDs using Framework Manager then you still would be building cubes using an externalized file.
The IQD’s can be replaced using Report Studio Report or a Framework Manager Package. In both the cases the source has to be pre-validated to avoid cross joins and other resource intensive sql.
Using Report Studio Reports as source for Transformer Cubes
While working with RS reports, we should keep in mind that we should not use objects like crosstab, repeaters etc. We can only work with LIST type reports.
Pros:
-The report would have been tested and simplified.
-Any change or update to the query can be managed using Framework Manager or Report Studio.
-Complex Calculations can be performed
Cons:
- Transformer only picks the fields that are in the report layout and not all those in Query.
- If the query is not dedicated for transformer use then we could be querying unnecessary columns. Layout calculations and prompts would have to be modified
-Reports specifications have to be updated if we need to add new fields to the transformer model
-We would have to ungroup columns and any counts based on grouped values would be lost too
-If the Transformer is based on ( structured and Fact queries) we would have to create multiple queries
Relational Package as a source for Transformer Cubes:
Pros:
- Importing relational sources gives more flexibility to the transformer modeler.
Cons:
-If the package is not designed well and requires updates
- Cross joins could be executed, if for example a single package with multiple namespaces was published. We could be creating dimensions from completely different namespaces.
DMR Packages as source for Transformer Cubes:
Pros:
- DMR packages offer visibility and predictability
- Developers can import dimensions from DMR package directly into the Transformer dimension map area.
-This make sure that dimension structure is standardized across the company
Cons:
-If a particular Transformer model requires a different hierarchy then we will have to maintain multiple versions of the dimension in FM
-We have to make sure Measures are allocated to all the lowest levels in all dimensions
-It takes more modeling resource/time as we have to generate dimensions and star groupings from Relational schemas.
The reasons for moving away from IQD’s are:
1. IQD’s are files and hence should be avoided
2. You might need to maintain Impromptu licenses
3. If you have created IQDs using Framework Manager then you still would be building cubes using an externalized file.
The IQD’s can be replaced using Report Studio Report or a Framework Manager Package. In both the cases the source has to be pre-validated to avoid cross joins and other resource intensive sql.
Using Report Studio Reports as source for Transformer Cubes
While working with RS reports, we should keep in mind that we should not use objects like crosstab, repeaters etc. We can only work with LIST type reports.
Pros:
-The report would have been tested and simplified.
-Any change or update to the query can be managed using Framework Manager or Report Studio.
-Complex Calculations can be performed
Cons:
- Transformer only picks the fields that are in the report layout and not all those in Query.
- If the query is not dedicated for transformer use then we could be querying unnecessary columns. Layout calculations and prompts would have to be modified
-Reports specifications have to be updated if we need to add new fields to the transformer model
-We would have to ungroup columns and any counts based on grouped values would be lost too
-If the Transformer is based on ( structured and Fact queries) we would have to create multiple queries
Relational Package as a source for Transformer Cubes:
Pros:
- Importing relational sources gives more flexibility to the transformer modeler.
Cons:
-If the package is not designed well and requires updates
- Cross joins could be executed, if for example a single package with multiple namespaces was published. We could be creating dimensions from completely different namespaces.
DMR Packages as source for Transformer Cubes:
Pros:
- DMR packages offer visibility and predictability
- Developers can import dimensions from DMR package directly into the Transformer dimension map area.
-This make sure that dimension structure is standardized across the company
Cons:
-If a particular Transformer model requires a different hierarchy then we will have to maintain multiple versions of the dimension in FM
-We have to make sure Measures are allocated to all the lowest levels in all dimensions
-It takes more modeling resource/time as we have to generate dimensions and star groupings from Relational schemas.
Thursday, July 23, 2009
Free End User Training - Cognos Query Studio
We are pleased to announce a FREE online training in Cognos Query Studio.Query studio is a Cognos tool that can whip up adhoc reports and analysis for users by users.
Introduction to Query Studio
Examine Query Studio and its interface
Add and save data to ad hoc reports
View data by using appropriate charts
Create list, grouped list, and crosstab reports
Filter and sort your reports
Create detail and summary calculations
Move columns and reorder columns
New features in Cognos8.4 Query Studio
Best Practices and optimization of queries
Please mark your calendars for August 21st 2009 between 10:00-11:00 AM. Please email support@softpath.net to confirm your seat and receive a copy of the training documentation.
Introduction to Query Studio
Examine Query Studio and its interface
Add and save data to ad hoc reports
View data by using appropriate charts
Create list, grouped list, and crosstab reports
Filter and sort your reports
Create detail and summary calculations
Move columns and reorder columns
New features in Cognos8.4 Query Studio
Best Practices and optimization of queries
Please mark your calendars for August 21st 2009 between 10:00-11:00 AM. Please email support@softpath.net to confirm your seat and receive a copy of the training documentation.
Wednesday, July 15, 2009
Pointers while Integrating/migrating- M&A scenario
Here are some general pointers to be kept in mind, while planning migrations, upgrades, conversions and the likes. Most of these recommendations can be made only after a thorough understanding of client systems architecture, maturity of their BI systems and their future plans. Anyways here is an appetizer.
1. Assess your architecture
2. Huge performance gains can be achieved by having dedicated servers for reporting services, batch report services, Agent and monitoring services and presentation services respectively.
3. Gateways
ISAPI is most recommended gateway. Also improve performance by adding multiple gateways.
Especially, if you have multiple locations for your servers
4. Perform Capacity Planning Exercise. Most BI tools offer pre built exercises.
5. If switching Security namespaces, you will encounter a lot of accessibility issues and will not be able to export personal content
6. If you have external vendors/clients accessing your BI portal the firewalls can cause accessibility issues. You will have to configure your firewalls and gateways to support multiple dispatchers.
7. Consider virtualization of your servers to cut costs
8. If in the near future, you plan on merging data sources multiple companies, you might have to plan back up of existing Models, packages and data source connections. Also, perform a step by step deployment into new environments or domains.
Some Challenges you might come across are:
1. Accessing cross company portals. Are you going to have 2 portals? Are you going to add users in two domains?
2. If development and support teams are in Domain A and have to support users in domain B. Developers and support Team, ticket management has to work across domains with additional privileges. This might not seem like huge liability but it can be time consuming to get it up and running
3. There will be a need for more licenses and appropriate capabilities
4. You will have to assess he hardware requirements and configuration
However, this is a very good time and opportunity to introduce new BI technologies and practices.
- You could start a BICC ( BI Competency Center)
- Upgrade to newer versions of the tools
1. Assess your architecture
2. Huge performance gains can be achieved by having dedicated servers for reporting services, batch report services, Agent and monitoring services and presentation services respectively.
3. Gateways
ISAPI is most recommended gateway. Also improve performance by adding multiple gateways.
Especially, if you have multiple locations for your servers
4. Perform Capacity Planning Exercise. Most BI tools offer pre built exercises.
5. If switching Security namespaces, you will encounter a lot of accessibility issues and will not be able to export personal content
6. If you have external vendors/clients accessing your BI portal the firewalls can cause accessibility issues. You will have to configure your firewalls and gateways to support multiple dispatchers.
7. Consider virtualization of your servers to cut costs
8. If in the near future, you plan on merging data sources multiple companies, you might have to plan back up of existing Models, packages and data source connections. Also, perform a step by step deployment into new environments or domains.
Some Challenges you might come across are:
1. Accessing cross company portals. Are you going to have 2 portals? Are you going to add users in two domains?
2. If development and support teams are in Domain A and have to support users in domain B. Developers and support Team, ticket management has to work across domains with additional privileges. This might not seem like huge liability but it can be time consuming to get it up and running
3. There will be a need for more licenses and appropriate capabilities
4. You will have to assess he hardware requirements and configuration
However, this is a very good time and opportunity to introduce new BI technologies and practices.
- You could start a BICC ( BI Competency Center)
- Upgrade to newer versions of the tools
Tuesday, June 16, 2009
Master Data Management
Authored by:Chintan
What is Master Data? Most of the IT organizations have their data dispersed and stored at various locations. But the data is shared and used by several of the applications that make up a data warehouse. For example, an ERP system has data from Customer Master, Item Master and Account Master. The master data is one of the key assets of any company.
Need of Master Data Management: Since master data is used by multiple applications, an error in master data can cause errors in all the applications that use it. For example, all important documents, bills, checks are sent to the wrong person because of an incorrect address in Customer Master. Similarly, an incorrect price in Item Master can be a marketing disaster for an organization. An incorrect account number in an Account Master can lead to huge fines. To overcome such hazards, maintaining high quality and consistent set of master data is necessary for all organizations.
What is Master Data Management (MDM)? The technology, tools and processes required to create and maintain consistent and accurate lists of master data is known as Master Data Management. MDM is a continuous, iterative process. There are many factors considered for MDM which involves requirements, priorities, resource availability, time frame and the size of the problem.
MDM Life Cycle: MDM project involves many stages as follows:
1- Identify sources of master data.
2- Identify the producers and consumers of the master data.
3- Collect and analyze metadata about master data.
4- Appoint data stewards.
5- Develop the master-data model.
6- Choose a toolset.
7- Finalize and receive approval for the process.
8- Design and implement the process.
9- Test the master data.
10- Modify the producing and consuming systems.
11- Implement the maintenance processes.
Conclusion: In recent times, creating and maintaining accurate and complete master data has become a business imperative. Both large and small businesses must develop data maintenance and governance processes and procedures, to obtain and maintain accurate master data.
What is Master Data? Most of the IT organizations have their data dispersed and stored at various locations. But the data is shared and used by several of the applications that make up a data warehouse. For example, an ERP system has data from Customer Master, Item Master and Account Master. The master data is one of the key assets of any company.
Need of Master Data Management: Since master data is used by multiple applications, an error in master data can cause errors in all the applications that use it. For example, all important documents, bills, checks are sent to the wrong person because of an incorrect address in Customer Master. Similarly, an incorrect price in Item Master can be a marketing disaster for an organization. An incorrect account number in an Account Master can lead to huge fines. To overcome such hazards, maintaining high quality and consistent set of master data is necessary for all organizations.
What is Master Data Management (MDM)? The technology, tools and processes required to create and maintain consistent and accurate lists of master data is known as Master Data Management. MDM is a continuous, iterative process. There are many factors considered for MDM which involves requirements, priorities, resource availability, time frame and the size of the problem.
MDM Life Cycle: MDM project involves many stages as follows:
1- Identify sources of master data.
2- Identify the producers and consumers of the master data.
3- Collect and analyze metadata about master data.
4- Appoint data stewards.
5- Develop the master-data model.
6- Choose a toolset.
7- Finalize and receive approval for the process.
8- Design and implement the process.
9- Test the master data.
10- Modify the producing and consuming systems.
11- Implement the maintenance processes.
Conclusion: In recent times, creating and maintaining accurate and complete master data has become a business imperative. Both large and small businesses must develop data maintenance and governance processes and procedures, to obtain and maintain accurate master data.
Wednesday, June 10, 2009
Data Quality
In 2003 the Data Warehousing Institute calculated that bad data quality leads to a whopping loss of, approx $600 billion annually.
Data Quality Improvement is the processes and technologies involved in ensuring the conformance of data values to business requirements and acceptance criteria.
The reasons that adversely affect the data quality are:
Legacy Systems and data: Legacy Systems may/may not have validations in built into them. Legacy Systems tend to have redundant data, composite keys and referential integrity issues.
Application Evolution : Applications evolve over time and the data entry operations, client and server side validations are often overlooked resulting in bad data quality.
System Work-Around: More often than not, immediate results and often temporary measures are deployed to meet time deadlines or technology limitations.
Time Decay: The best of the systems cannot stand the test of the time. What better example than Y2K bug. Data quality deteriorates with time.
Lack of common data standards: Companies do not always invest time and resources into creating best practices, standards and checklists. Simple tasks such as having universal naming conventions can improve quality of the data.
Data Entry issues: Data entry issues are top1 reason for adversely affecting the quality of the data. If data entry is performed by customers or web based users, it is most likely that junk and misplaced information will be gathered. Even internal data entry operations are compromised because of the ‘remarks’ or ‘comments’ sections.
So how can this data be cleared up?
DIY: Do It Yourself by looking into databases, forms, applications etc. Of course, this is not the best choice. But is a beginner’s step that can lead you to a roadmap for data cleansing.
Invest in Data Cleansing Tools: Outsource to who can do it best. Yes, now we are talking business. Invest in identifying the suitable tools in the market. Here are some:
· Informatica Power Center
· Trillium Software
· Business Objects Data Integrator
· Data Flux
So, going forward how do you prevent rather than cure? Here are some ideas gathered by us.
Data Profiling – analyze the date for correctness, completeness, uniqueness, consistency, and reasonability. This must be done in the order of column profiling, dependency profiling, and then redundancy profiling.
Data Cleansing – Detect and correct corrupt or inaccurate records from a record set, table, or database. The common methods are parsing, data transformation, duplicate elimination, and many statistical methods.
Data Defect Prevention –Set up a data governance group that will take control and responsibility of the various databases and enforce/introduce data quality rules. They will conduct regular audits and data cleansing programs. They also have to take charge of the training of data entry and other personnel.
Data Quality: Authored by Vivek and Devi. Cleansed by Vai :-)
Data Quality Improvement is the processes and technologies involved in ensuring the conformance of data values to business requirements and acceptance criteria.
The reasons that adversely affect the data quality are:
Legacy Systems and data: Legacy Systems may/may not have validations in built into them. Legacy Systems tend to have redundant data, composite keys and referential integrity issues.
Application Evolution : Applications evolve over time and the data entry operations, client and server side validations are often overlooked resulting in bad data quality.
System Work-Around: More often than not, immediate results and often temporary measures are deployed to meet time deadlines or technology limitations.
Time Decay: The best of the systems cannot stand the test of the time. What better example than Y2K bug. Data quality deteriorates with time.
Lack of common data standards: Companies do not always invest time and resources into creating best practices, standards and checklists. Simple tasks such as having universal naming conventions can improve quality of the data.
Data Entry issues: Data entry issues are top1 reason for adversely affecting the quality of the data. If data entry is performed by customers or web based users, it is most likely that junk and misplaced information will be gathered. Even internal data entry operations are compromised because of the ‘remarks’ or ‘comments’ sections.
So how can this data be cleared up?
DIY: Do It Yourself by looking into databases, forms, applications etc. Of course, this is not the best choice. But is a beginner’s step that can lead you to a roadmap for data cleansing.
Invest in Data Cleansing Tools: Outsource to who can do it best. Yes, now we are talking business. Invest in identifying the suitable tools in the market. Here are some:
· Informatica Power Center
· Trillium Software
· Business Objects Data Integrator
· Data Flux
So, going forward how do you prevent rather than cure? Here are some ideas gathered by us.
Data Profiling – analyze the date for correctness, completeness, uniqueness, consistency, and reasonability. This must be done in the order of column profiling, dependency profiling, and then redundancy profiling.
Data Cleansing – Detect and correct corrupt or inaccurate records from a record set, table, or database. The common methods are parsing, data transformation, duplicate elimination, and many statistical methods.
Data Defect Prevention –Set up a data governance group that will take control and responsibility of the various databases and enforce/introduce data quality rules. They will conduct regular audits and data cleansing programs. They also have to take charge of the training of data entry and other personnel.
Data Quality: Authored by Vivek and Devi. Cleansed by Vai :-)
Subscribe to:
Posts (Atom)
