Data to Decisions

Integration, Automation, and Data Analytics

Flow Integration - Automating Excel to Oracle

The Challenge 4DIQ’s client, a global accounting firm, used an Oracle Database to store and ac

The Challenge

4DIQ’s client, a global accounting firm, used an Oracle Database to store and access their client’s raw financial and transaction data for audit purposes. Each day this client receives an average of forty Excel files from various clients. The number of Excel files received is substantially greater during end of quarter processing.

Their current application suite did not meet their unique needs. The client needed a data mart solution that:

  1. Captured and recorded the metadata within Excel Files sent to them by clients
  2. This data included o File information including the name, date created, date modified, and size
    • The calculation of a min, max, range, variance, standard deviation, and data outliers for each numeric column within each Worksheet
    • The calculation of min, max, range, and outliers for each date column within each worksheet
    • The number of missing dates for each date column within each worksheet
    • The number of missing, empty, or null text values for each text columns within each worksheet
    • The max and min length of each text column within each worksheet
  1. Store the metadata for each client’s Excel files and each worksheet within those files in an Oracle database to form a catalog of the data provided by each client
  2. Create a table for each worksheet within their Oracle database and insert the data values.
  3. Provide aggregated views of the metadata and raw data for
  4. Automate the entire process

Current State Analysis

Overview

The client’s current state solution involved manually collecting and calculating the metadata associated with each file. First, file attribute information was manually recorded in a separate Excel spreadsheet or local Access database. Second, each Excel file was opened and the column metadata was calculated then manually rekeyed or copied to the metadata spreadsheet or Access database.  After recording the metadata, the client then used an Oracle ETL process to load the both the metadata and Excel file worksheet data. 

Current State Process

The process employed by the client to capture required metadata and load the raw data:

  1. The client periodically checked a common directory for any new or modified Excel files
  2. Any new or modified files were opened individually and sequentially
  3. A new Excel Spreadsheet was created to hold the metadata for each new file or an existing spreadsheet was located and opened for any modified file
  4. The file metadata was noted and recorded in the metadata spreadsheet using the file properties displayed by Windows Explorer
  5. The Excel file was then opened and visually inspected
  6. A set of common functions for calculating text, numeric, and date metrics was copied and pasted to the end of each column.
  7. The metrics were manually copied or rekeyed to the metadata spreadsheet.
  8. The metadata spreadsheet was formatted and exported to a comma separated file for use by the Oracle ETL process.
  9. The data in each worksheet was exported to a comma separated file for use by the Oracle ETL process
  10. The ETL processes were run and the data was uploaded to the Oracle database
  11. A notification was sent to the assigned data analyst
  12. Since each set of data had a unique structure, the data analyst queried the database tables to visually inspect the data
  13. The data analyst then used a combination of SQL queries and data import routines to capture the data for analysis purposes.

 

This process was repeated on a daily basis. It was an extremely time intensive requiring approximately 30 minutes to process each Excel file. This consumed about 30 hours per day or, on average, 150 hours per week. At an assumed average cost of $30 / £18 per man hour, the average weekly cost was $4,500 or £2,848. 

The current state processes were time consuming, costly, and extremely error prone. 4DIQ provided a future state analysis and proposal to employ a Flow DI solution that would significantly lower costs, decrease errors, and reduce resource requirements.

Future State Analysis

 

Overview

4DIQ provided a proposal that employed a custom Flow Workflow to automate the process end-to-end. The proof of concept design took less than two weeks and the solution implementation was completed within five days.

Future State Process

The Flow Workflow steps that comprise the solution are outlined below:

  1. Create a file monitoring task that raises an event whenever a new file is added to a client file repository or an existing file modified.
  2. Upon the occurrence of the a file event, import two sets of data for each file:
    1. The file attribute data
    2. The data from file’s worksheets
  3. Normalize the file attribute data
  4. Calculate the required metrics for each data column based upon the type of data: numeric, text, or date.
  5. Normalize the column data by mapping common field values to predefined names. For example, QTY to Quantity and AMT to Amount. This provides normalized names for the eighty percent of the common fields used by the data analysts
  6. Query the Oracle database for an existing client
  7. If a client exists
    1. Execute SQL to create required tables
    2. Execute an upsert of the metadata and worksheet data into the tables
  8. If no client exists
    1. Execute a client record insert
    2. Execute SQL to create required tables
    3. Execute an upsert of the metadata and worksheet data into the tables
  9. Execute an email notification task to the responsible individual notifying them that the data is available.
  10. Output the normalized data in a format used by a third-party system for further analysis.

.

Results

 

Implementation

The replacement of this complex manual process took less than three weeks to design, validate, and implement. Creating a custom coded integration workflow process from the ground up using traditional methods would have been costly and time consuming. In addition, such a solution would have little future reuse.

Post Implementation Changes

With this solution in place, the client requested that it be modified to include comma separated files.  Through the web portal, the client was able to launch the Flow development environment to modify the workflow.  

Eliminated 100 percent of manual processes

Integration using a Flow Workflow eliminated all of the manual processes that were previously required to accomplish this critical task.

Strengthened Oversight

Tracking client files and the metadata associated with them enabled the firm to easily inventory their client data assets. In addition, the normalization of analysis data and notification of responsible parties enabled managers to more effectively track the status of projects and better allocate resources. 4DIQ designed a dashboard that allowed managers to access key metrics and KPIs such as the number of files received and processed per day, summaries of data metrics, and a list of each client’s data files.

ROI

Prior to implementation of the Flow Solution, the firm was spending an estimated $234,000 or approximately £148,000 on manual processes. The Flow solution provided a cost reduction of over eighty five percent. 

Customer Data Integration Solutions For Financial Services Firms

  Customer Data Integration Solutions For Financial Services Firms Customer data integration le
 

Customer Data Integration Solutions For Financial Services Firms

Customer data integration leads to improved customer intelligence which enables financial service firms to gain a more complete and timely measure of the value of customer relationships across lines of business. However, a measure of the value of customer relationships that relies exclusively upon traditional, internal sources of customer data may not reflect their true value.  New sources of customer data from social, mobile, and cloud based sources may more accurately reflect the underlying relationship value.

Leveraging these new sources of customer data in combination with traditional sources can lead to increased profitability through improved customer analysis, enhanced customer interaction, better customer feedback, and more effective customer relationship management.

4DIQ's customer data integration solutions leverage new and traditional sources of customer data and add value by helping to improve customer segmentation, provide a more complete view of multiple customer dimensions, and enable more timely delivery of customer data. The benefits of 4DIQ’s customer data integration help financial firms to better capitalize all their sources of customer data by enabling them to improve their marketing mix analysis, optimize marketing operations, and perform more accurate analysis of marketing performance.             

Customer Data Integration Leads to Improved Customer Intelligence

Banks, insurance and securities firms, lenders, and other financial institutions struggle with leveraging existing systems that do not share information with each other very well. This problem only worsens when business processes change and new systems are added to meet evolving requirements. A key challenge of integrating these diversified and distributed systems into a coherent, efficient infrastructure is creating a comprehensive view of the customer.

A comprehensive view of the customer answers critical business questions such as:

  • What are the most profitable opportunities for growth?
  • Which marketing actions will have the greatest impact?
  • What activities will have the maximum impact across lines of business?

Fortunately, 4DIQ has significant experience in overcoming the comprehensive customer view challenge for companies in the financial services industry.  4DIQ data integration solutions for financial services deliver a unified view of data across an entire organization, from both internal and external sources, maximizing the value of customer information. These solutions streamline business processes, promote efficiency, meet compliance demands, strengthen relationships with customers, and reduce operational costs.

New Sources of Customer Data

The Financial Services industry is in the midst of revolutionary new technology trends with the emergence of social, mobile, and cloud computing. These emerging trends require businesses to innovate and bridge the gap between new technologies that offer opportunities for growth and the maintenance of legacy system. Financial Services organizations that bridge this gap will be able to capitalize on these changes through more effective customer intelligence and position themselves for future success, enabling them to gain a competitive advantage.

Social Media

Social Media like blogs, Facebook, Google+, LinkedIn, and Twitter have grown increasingly popular as consumers are increasingly turning to these new channels for information and advice. Financial Services organizations that harness the power of social media can learn more about their customers, foster longlasting relationships with their customers, and reach out to their customers. Effective use of social media means leveraging additional communication channels so that companies can listen and relate to their community.

4DIQ data integration solutions empower Financial Services organizations to proactively engage customers using social media for product development and innovation, sales and marketing activities, public relations campaigns, and customer service initiatives. Financial Services organizations can gain a competitive edge by tapping in to these new social media channels. 

Mobile Devices

Customer interaction via mobile devices like smartphones and tablets plays an increasingly important new role as more consumers are turning to the convenience of these devices for their financial service needs. Financial Services organizations that are taking advantage of the ubiquitous nature of mobile devices have access to an important channel to reach and engage customers at home, work, or on-thego. Mobile devices streamline and simplify access, allowing customers to quickly connect to a variety of financial services.

4DIQ data integration solutions enable Financial Services organizations to get right information into the hands of the customer by bringing data together from different applications and systems. Financial Services organizations can leverage mobile devices as extensions of existing financial services delivery.

Cloud Computing

Cloud computing is fast evolving into a commercially viable alternative for businesses seeking additional value in technology. Financial Services organizations that are abandoning their servers in favor of the cloud are reducing IT costs and freeing up more time for customer service.  Cloud computing offers the benefits of scalability, flexibility, accessibility, and fast implementation. 

 4DIQ data integration solutions allow Financial Services organizations to reap the benefits of cloud computing by integrating cloud services with legacy infrastructure. The cloud naturally supports mobility, enabling internal resources to access email, software, and databases remotely. Financial Services organizations can utilize cloud computing to increase productivity and customer focus without negatively impacting the bottom line.

Leveraging New Sources of Customer Data

Attempts to leverage customer information for increased profitability have relied upon integration of customer data from a mix of operational systems. These systems were designed to support specific business functions, not to work together seamlessly. While the objective is to increase profitability through better customer intelligence, the reliance upon operational systems has often had the opposite effect. Dispersed, disconnected operational systems typically require manual integration processes which are complex and inefficient, frustrating attempts to leverage customer information for increased profitability.

4DIQ data integration solutions support strategy and planning by improving four critical customer intelligence functions:

  • Customer analysis,
  • Customer interaction,
  • Customer feedback, and
  • Customer relationship management

4DIQ enables Financial Services organizations to successfully increase their return on customer relationships by creating a comprehensive picture of the customer with current information from all internal and external sources customer information. 4DIQ data integration solutions can provide access to an integrated, 360-degree view of the customer allowing companies to integrate customer intelligence into their evolving business strategy and planning.

The Benefits of 4DIQ’s Customer Data Integration Solution

As new sources of customer data become available every day, the acquisition and integration of data to support timely and accurate customer analysis will increasingly extend beyond the integration of existing systems. 4DIQ's data integration solutions can combine existing sources of customer data with new, emerging sources to enable financial service firms to better capitalize on customer data through:

  • Improved customer segmentation
  • Dynamic, in-memory creation of multiple customer dimensions
  • The elimination of the need to build and maintain expensive data warehouses
  • More timely delivery of customer data directly to decision makers via dashboards and reports

Data integration that supports timely and accurate segmentation of customers allows Financial Services firms to focus on the customers who matter most. Businesses are better equipped to identify profitable customer segments across lines of business. In addition, timely and accurate customer segmentation provides more effective one-to-one marketing efforts.

Traditional methods of data integration to support customer segmentation relied upon the creation of data warehouses to support the production of multi-dimensional reports. 4DIQ’s data integration technology enables fast, in memory creation of dimensional data. This is not only less expensive than traditional data warehouse approaches but, in many cases, more powerful and flexible.

Capitalizing on New Sources of Customer Data

Capitalizing on new sources of customer data requires timely integration to gain accurate insight into the profitability of customer relationships across lines of business. This accurate insight is manifest through improved marketing mix analysis, optimized marketing operations, and more accurate analysis of marketing performance. 

Improved Marketing Mix Analysis

4DIQ's data integration solutions increase the accuracy and effectiveness of your marketing mix analysis through:

  • 4DIQ’s Configure-Not-Code™ service delivery for rapid integration and processing of new data sources
  • Configurable workflows that support robust data scrubbing and cleansing
  • Customizable data integration workflows that incorporate advanced data analysis capabilities including descriptive and predictive statistics
  • Workflows that use transformations, rules, events, and analytic functions to push required data to reports and scorecards for use by decision makers

Data integration solutions from 4DIQ enable businesses to more accurately and effectively utilize existing and new sources of marketing data. In addition, higher data quality improves forecast accuracy and can lower the risk of committing resources to the wrong marketing mix. More accurate and effective data integration can therefore increase competitive advantage through faster, more responsive, and better targeted marketing mix analysis.

Optimized Marketing Operations

The ability to rapidly integrate heterogeneous sources of data helps executives and managers to optimize marketing operations. Traditional approaches to accessing and integrating data required to support marketing operations are costly and time consuming. 4DIQ's innovative data integration solutions impact marketing operations by enabling:   

  • Rapid access to data from multiple heterogeneous sources for comprehensive, customized reporting
  • The ability to deliver timely, accurate data to custom or pre-built marketing dashboards
  • The creation and management of custom marketing data integration workflows to support meaningful metrics and KPIs
  • The linking of data integration workflows to the performance specific campaigns and related opportunities

Data integration solutions from 4DIQ can help to optimize marketing operations in a number of ways. Our solutions enable the rapid, cost-effective reuse of existing data sources, assets, and resources; In addition, our data integration solutions allow faster creation, deployment, and rapid feedback on the performance of marketing campaigns. Finally, 4DIQ data integration solutions provide better tracking of ROI by strengthening the links between campaigns and opportunities.

Accurate Analysis of Marketing Performance

Marketing performance analysis is enhanced by 4DIQ's data integration solution which allows:

  • The rapid integration and aggregation of data to support dynamic metrics and KPIs tied to specific business strategies and goals
  • The creation of customized, in-memory, multi-dimensional data aggregations for rapid analysis of customer and marketing data
  • The fast creation of custom workflows to integrate cost and revenue data across lines-of-business

This improves the alignment of strategies and goals with campaigns and opportunities. Increases the ability to measure and analyze marketing effectiveness and optimizes resource allocation.

Improved Customer Relationship Management (CRM)

CRM software like Salesforce, Microsoft Dynamics CRM, Redtail CRM, and Junxure will be a key differentiator for successful businesses as objectives expand from retention to growth. Financial Services organizations that are creating a complete picture of the customer will deliver excellent customer service while selling additional product and services. CRM software continues to improve with new social networking capabilities, better analytical tools, increased automation, and lower-cost solutions. The realities of cloud computing, the accessibility of mobile devices, and the impact of social media are all positively affecting the CRM landscape.

4DIQ data integration solutions provide Financial Services organizations with an integrated view of the customer, by connecting customer information from disparate sources with the CRM system. Financial Services organizations can maximize not only their CRM investment, but also their relationships with each customer.

Conclusion and Summary

Financial Services organizations need to solve the problems created by the explosion of customer information and demands, in an environment where both resources and budgets are constrained. Businesses that overcome their customer integration challenges stand a much greater chance of creating opportunities to sell new products and services to customers, and surpass the competition.

4DIQ replaces business-as-usual integration solutions with a new and fundamentally different approach. Managed Data Integration Services from 4DIQ provide cost-effective end-to-end data integration for Financial Services organizations, allowing them to streamline processes, improve the customer experience, and save money all with greater security and visibility.

Flow for Professional Services Firms

The Challenge of Integration and Automation in Professional Services Increased efficiency represents

The Challenge of Integration and Automation in Professional Services

Increased efficiency represents a unique challenge for professional services firms. The day-to-day pressure of serving client needs leaves little time to focus upon optimizing your firms overall efficiency. Manual processes begun as one-time shortcuts evolve into accepted business practice.

Critical data begins to accumulate in spreadsheets and files that are locked away in departmental servers or files located on the computers of key employees. Over time, the proliferation of critical data and manual integration processes slowly saps productivity. Small problems manifest as projects are delayed, operational errors increase, and expenses grow.

Difficult and Challenging Issues

These issues are challenging to solve because the proliferation of data coupled with lack of integration and automation make solutions difficult to pinpoint or define. Moreover, lack of integration and automation make new business productivity software more expensive and challenging to implement. In addition, maintaining financial controls, establishing baseline productivity measures, and obtaining accurate business analytics becomes increasingly time consuming and expensive.

How Flow Overcomes the Integration and Automation Challenge

  • Universal Data Access enables plug-and-play connections to any source of data from files to spreadsheets and databases.
  • Distributed data integration ensures that Flow can import data from where it resides.
  • Workflows automate the tasks required to import, process, and integrate disparate sources of data.
  • Cloud based orchestration enables access across locations and enables integration of internal and external sources of data
  • Centralized management and monitoring of automated workflows require less time and fewer resources to consolidate critical business data.
  • Powerful integration and automation processes can be quickly designed and deployed using the Flow Workflow Editor.
  • Workflows can model, implement, and automate any required business process. The result is rapid and cost-effective delivery of critical information.

Moving Beyond the Challenge of Integration and Automation

With powerful reporting, dashboards, and analytics capabilities, Flow can help your firm move beyond integration and automation. With Flow, your firm can realize the power of business analytics to:

  • Deliver meaningful performance insights to firm managers
  • Evaluate competitive benchmarks and overall market trends
  • Discover opportunities that offer the lowest risks and highest returns
  • Manage risk
  • Manage and optimize business processes
  • Gain a multidimensional view of your clients
  • Enable employees to become more productive and concentrate on activities that produce revenue

Benefits for Professional Services Firms
Flow professional service users report benefits such as:

  • Better knowledge of their business
  • Faster response to proposal requests
  • Increased sales pipelines
  • More accurate cost and revenue forecasts

Cloud Deployment Considerations for Financial Services Firms

Overview The adoption of cloud computing and related technologies has been slow to take hold within

Overview

The adoption of cloud computing and related technologies has been slow to take hold within the financial services industry. This is more so the case when looking at the adoption rate among small to mid-sized financial services firms. These firms include wealth management departments of banks, trust, financial planners, and Registered Investment Advisors. This post takes a look at cloud computing, three types of cloud deployment models, and key considerations regarding which deployment to choose.

Cloud Computing Definition

Despite all the hype, it is surprising to discover the number of business people who still ask me what cloud computing is. So, I'll start off by offering the National Institutes of Standards and Technology's definition of cloud computing. Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Key Considerations

In the financial services industry, there are several key considerations for adopting a cloud deployment model:

  • Regulatory, compliance, and risk
  • Client experience
  • Integration of on-premise, cloud-base, and external data and application services

Cloud Deployment Models

There are three types of cloud deployment models:

  1. Public Cloud
  2. Private Cloud
  3. Hybrid Cloud

1. Public Cloud

Services Delivered

Infrastructure as a Service (IaaS) - The service provider owns the equipment (storage, hardware, servers and networking components) and is responsible for facilities, network infrastructure, maintenance, operations, and support. The service provider maintains and balances the cost of excess capacity. The IaaS client provisions services on a pay per-use basis, i.e. storage, computing power, and network bandwidth.

Platform as a Service (PaaS) - Platform as a service (PaaS) is a category of cloud computing services that provide a computing platform and a solution stack as a service. In the classic layered model of cloud computing the PaaS layer lies between the SaaS and the IaaS layers.

Software as a Service (SaaS) - Software as a service, is a software delivery model in which software and its associated data are centrally hosted and accessed over the public Internet by users via a thin client, typically a web browser.

Benefits

  • Easy and inexpensive set-up because the cloud service provider is responsible for supplying required infrastructure
  • Self-service on-demand provisioning of resources
  • Universal network access allows organizations to pool resources independent of their location
  • Scales to meet business needs - excess capacity provides clients with rapid elasticity allowing them to expand and contract in response to business demand

Public Cloud Suitability for Financial Services

Public clouds are well-suited to meet financial service industry needs that are low risk, scalable, and have predictable workloads. They also allow financial service firms to experiment and gain familiarity with new technologies for service delivery at a low cost. Public clouds adoption within the financial services industry is currently centered around simple and repetitive workloads such as data and document storage, email, computing services, and SaaS applications such as CRM, HR, and collaborative applications.

2. Private Cloud

Also called an internal or corporate cloud, it is a proprietary computing architecture that delivers cloud services to a limited number of people behind a firewall.

Services Delivered

Infrastructure as a Service (IaaS) - The organization owns the equipment and is responsible for facilities, network infrastructure, maintenance, operations, and support.

Platform as a Service (PaaS) - PaaS is rarely delivered from a private cloud.

Software as a Service (SaaS) - Software as a service, is a software delivery model in which software and its associated data are centrally hosted behind the organizations firewall and accessed by internal users, which can include partners and customers, over the organizations private network. User access is via a thin client, typically a web browser.

Benefits

  • Security and compliance
  • Lower operational risk
  • Easier to configure and re-configure
  • Lower average cost per deployed application

Private Cloud Suitability for Financial Services

Public clouds are well-suited to meet financial service industry requirements for low risk exposure, stringent security and compliance measures, and the need to configure and re-configure infrastructure and software assets at a lower cost. They allow financial service firms to provision resources and provide services that require secure internal access.

3. Hybrid Cloud

A cloud computing environment in which financial service firm provides and manages certain resources in-house and others externally. A financial services firm might use a public cloud service to archive data and documents but while storage operational data, transactions, and sensitive customer information on-premise. This allows a financial services firm to use the scalable and cost-effectiveness of a public cloud computing environment without exposing mission-critical applications and data to third-party vulnerabilities.

Client Centric Cloud Strategy

Financial service firms face high requirements for security, predictability, and compliance. Therefore, all information technology resource including internal, outsource, cloud services, and SaaS applications should be managed as a single computing environment. Public, private, and hybrid cloud service delivery must be act as a single, seamless environment. The complexities of any environment must be abstracted away from the end client. There must be consistent methods and metrics for monitoring, governing, and measuring cloud services. A single, client centric service level should be established for all services that touch clients.

A financial service firm's cloud strategy should focus first and foremost on clients. The use of cloud services should provide customers with a safe, consistent and predictable experience. A hybrid strategy allows financial service firms to combine the benefits of public and private cloud services to deliver high-quality client services. However, hybrid cloud services mix on-premise and off-premise applications and data. Providing high-quality client services depends in large part on the integration of on-premise and cloud-based data. Data integration therefore becomes critical in a hybrid cloud environment.

Financial service firms evaluating cloud based deployment models must require vendors to:

  • Have a cohesive data integration strategy between on-premise, cloud-based, and external providers
  • Offer an SLA that provides for an integrated approach to data
  • Have a data quality model that meets the quality needs of customers

Four Key Benefits of Data Integration for Financial Services Firms

Financial service professionals often do not distinguish the terms 'information' and 'data'. However

Financial service professionals often do not distinguish the terms 'information' and 'data'. However, the business impact of new data sources available to the financial services industry make it imperative to understand the difference between these terms. Information, such as client reports, alerts, and recommendations, is the primary product produced by most financial services firms. Further, data, including market, client, and economic, is the raw material used to produce it. Therefore, data is one of the most important resources owned or used by financial services firms. The level of data integration employed by a firm is a good measure of the efficiency with which a firm produces their product. It is also a good indicator of a financial service firm's client service and product quality. This makes the level of data integration an important indicator of a firm's current and future health and profitability.

Unfortunately, merely mentioning the words data or integration is often enough to make many financial service professionals abruptly refer you to the nearest available IT employee. I am, of course, exaggerating to make a point, which is: financial services professionals should pay close attention to discussions of data and integration. Outlined below are four key reasons data integration deserves attention in financial services:

1. Increased New Business Opportunities & Client Retention Rates

In the near future, the competition for new business will be won by firms able to create information that serves client needs as understood from a complete, so called 360, view of a client. Since the competition amongst firms for client assets is a zero sum game, information provided to clients based upon a complete and accurate view of their needs will also greatly impact client retention rates. Firms that have efficient data integration capabilities will be able create and provide a better information product to their customers. That is, they will be able to offer timely, high-quality information that is uniquely tailored to the needs of their current and prospective clients. To generate this information, these firms will need to efficiently integrate data from a large number of heterogeneous sources. These include internal operational data, third-party data, and newer data sources such as social media.

Unfortunately, merely mentioning the words data or integration is often enough to make many financial service professionals abruptly refer you to the nearest available IT employee. I am, of course, exaggerating to make a point, which is: financial services professionals should pay close attention to discussions of data and integration. Outlined below are four key reasons data integration deserves attention in financial services.

2. Increased Efficiency and Decreased Costs

There are two primary service delivery approaches implemented by financial service firms: best-of-breed and end-to-end. Each of these approaches creates unique integration complexities that increase costs. Those complexities include data duplication, required customization, and process inefficiency.

Data is a perishable commodity. That is, it is a raw material that possesses a limited shelf life. Therefore, rapid and efficient access to timely data reduces waste. Over time, the logistics of internal data integration have been made more complex due to data duplication arising from siloed best-of-breed applications or inflexible end-to-end solutions. The best-of-breed and end-to-end solution approaches have made internal data integration more difficult and complex. The addition of external data sources, such as social media, will add a new layer of complexity on top of this. Therefore, front, middle, and back-office technology will not translate into increased efficiency. More efficient use of existing technology primarily means more efficient data integration.

Data integration remains a complex process, so a light weight, low cost, and efficient data integration solution managed with the proper skill and expertise are needed to overcome this complexity. Such a solution, implemented as an overlay on existing technology, will greatly decrease costs.

3. Decreased Risk Exposure

Data integration efficiency directly impacts a financial service firm’s risk exposure. Complex integration, outdated data, and poor data quality lead to the production of inconsistent, outdated, or inaccurate information. This greatly increases a firm’s risk exposure.

Decreasing risk exposure requires timely and accurate data to be delivered both internally and externally. This enables informed decisions on the part of everyone involved in the financial services delivery process. It also insures that processes are consistent which results in consistent, compliant information provided to clients and regulators.

4. Increased Product Quality

Financial services firms’ primary product is information, and the quality of that product is directly impacted by the quality and timeliness of the underlying data.

Increased product quality requires financial service firms to have timely and efficient access to the most up-to-date and accurate data. In addition, product quality is increasingly defined by how well information provided to clients reflects their unique personal circumstances. This requires integration of data that is unique and customer centric. Data integration flexibility, therefore, is becoming extremely important.

Data integration has a profound impact upon the quality of product produced by financial service firms. Moreover, product quality will increasingly be measured by how well a firm delivers information that is tailored to unique client needs. As clients begin to expect information tailored directly to their needs, efficient, high value data integration services will become more of a business critical function for financial service firms.

Customer Data Integration Leads to Improved Customer Intelligence

Banks, insurance and securities firms, lenders, and other financial institutions struggle with lever

Banks, insurance and securities firms, lenders, and other financial institutions struggle with leveraging existing systems that do not share information with each other very well. This problem only worsens when business processes change and new systems are added to meet evolving requirements. A key challenge of integrating these diversified and distributed systems into a coherent, efficient infrastructure is creating a comprehensive view of the customer.

A comprehensive view of the customer answers critical business questions such as:

  • What are the most profitable opportunities for growth?
  • Which marketing actions will have the greatest impact?
  • What activities will have the maximum impact across lines of business?

Customer data integration solutions for financial services deliver a unified view of data across an entire organization, from both internal and external sources, maximizing the value of customer information. These solutions streamline business processes, promote efficiency, meet compliance demands, strengthen relationships with customers, and reduce operational costs.

New Sources of Customer Data

The Financial Services industry is in the midst of revolutionary new technology trends with the emergence of social, mobile, and cloud computing. These emerging trends require businesses to innovate and bridge the gap between new technologies that offer opportunities for growth and the maintenance of legacy system. Financial Services organizations that bridge this gap will be able to capitalize on these changes through more effective customer intelligence and position themselves for future success, enabling them to gain a competitive advantage.

Customer data integration solutions empower Financial Services organizations to proactively engage customers using social media for product development and innovation, sales and marketing activities, public relations campaigns, and customer service initiatives. Financial Services organizations can gain a competitive edge by tapping in to these new social media channels.

Mobile Devices

Customer interaction via mobile devices like smartphones and tablets plays an increasingly important new role as more consumers are turning to the convenience of these devices for their financial service needs. Financial Services organizations that are taking advantage of the ubiquitous nature of mobile devices have access to an important channel to reach and engage customers at home, work, or on-the-go. Mobile devices streamline and simplify access, allowing customers to quickly connect to a variety of financial services.

Customer data integration solutions enable Financial Services organizations to get right information into the hands of the customer by bringing data together from different applications and systems. Financial Services organizations can leverage mobile devices as extensions of existing financial services delivery.

Cloud Computing

Cloud computing is fast evolving into a commercially viable alternative for businesses seeking additional value in technology. Financial Services organizations that are abandoning their servers in favor of the cloud are reducing IT costs and freeing up more time for customer service. Cloud computing offers the benefits of scalability, flexibility, accessibility, and fast implementation.

Customer data integration solutions allow Financial Services organizations to reap the benefits of cloud computing by integrating cloud services with legacy infrastructure. The cloud naturally supports mobility, enabling internal resources to access email, software, and databases remotely. Financial Services organizations can utilize cloud computing to increase productivity and customer focus without negatively impacting the bottom line.

Leveraging New Sources of Customer Data

Attempts to leverage customer information for increased profitability have relied upon integration of customer data from a mix of operational systems. These systems were designed to support specific business functions, not to work together seamlessly. While the objective is to increase profitability through better customer intelligence, the reliance upon operational systems has often had the opposite effect. Dispersed, disconnected operational systems typically require manual integration processes which are complex and inefficient, frustrating attempts to leverage customer information for increased profitability.

Customer data integration solutions support strategy and planning by improving four critical customer intelligence functions:

  • Customer analysis
  • Customer interaction,
  • Customer feedback, and
  • Customer relationship management

Customer data integration enables Financial Services organizations to successfully increase their return on customer relationships by creating a comprehensive picture of the customer with current information from all internal and external sources customer information. Customer data integration solutions can provide access to an integrated, 360-degree view of the customer allowing companies to integrate customer intelligence into their evolving business strategy and planning.

Six Marketing Benefits of Customer Data Integration

As new sources of customer data become available every day, the acquisition and integration of data

As new sources of customer data become available every day, the acquisition and integration of data to support timely and accurate customer analysis will increasingly extend beyond the integration of existing systems. Customer Data Integration solutions can combine existing sources of customer data with new, emerging sources to enable businesses to better capitalize on customer data through:

1. Improved Customer Segmentation

Data integration that supports timely and accurate segmentation of customers allows firms to focus on the customers who matter most. Businesses are better equipped to identify profitable customer segments across lines of business. In addition, timely and accurate customer segmentation provides more effective one-to-one marketing efforts.

2. Dynamic Creation of Customer Dimensions

Traditional methods of data integration to support customer segmentation relied upon the creation of data warehouses to support the production of multi-dimensional reports. Customer Data Integration enables fast, in memory creation of dimensional data. This is not only less expensive than traditional data warehouse approaches but, in many cases, more powerful and flexible.

3. Timely Delivery of Customer Marketing Data

Capitalizing on new sources of customer data requires timely integration to gain accurate insight into the profitability of customer relationships across lines of business. This accurate insight is manifest through improved marketing mix analysis, optimized marketing operations, and more accurate analysis of marketing performance.

4. Improved Marketing Mix Analysis

Customer Data Integration solutions enable businesses to more accurately and effectively utilize existing and new sources of marketing data. In addition, higher data quality improves forecast accuracy and can lower the risk of committing resources to the wrong marketing mix. More accurate and effective Customer Data Integration can therefore increase competitive advantage through faster, more responsive, and better targeted marketing mix analysis. To improve marketing mix analysis, a Customer Data Integration solution should have:

  • The ability to rapidly integrate and process new data sources
  • Configurable workflows that support robust data scrubbing and cleansing
  • The ability to perform predictive and descriptive data analysis within data integration processes
  • The use of transformations, rules, events, and analytic functions to push required data to reports and scorecards for use by decision makers

5. Optimized Marketing Operations

Customer Data Integration can help to optimize marketing operations in a number of ways. Customer Data Integration enables rapid, cost-effective reuse of existing data sources, assets, and resources; In addition, Customer Data Integration allows faster creation, deployment, and rapid feedback on the performance of marketing campaigns. Finally, Customer Data Integration provides better tracking of ROI by strengthening the links between campaigns and opportunities.

The ability to rapidly integrate heterogeneous sources of data helps executives and managers to optimize marketing operations. Traditional approaches to accessing and integrating the customer data required to support marketing operations are often costly and slow.To help optimize marketing operations, a Customer Data Integration solution should:

  • Provide rapid access to data from multiple heterogeneous sources for comprehensive, customized reporting
  • Deliver both timely, accurate data to custom or pre-built marketing dashboards
  • Allow the creation and management of customer data integration workflows that provide meaningful marketing metrics and KPIs
  • Link data integration workflows to the performance specific campaigns and related opportunities

6. More Accurate Analysis of Marketing Performance

Improving the alignment of strategies and goals with campaigns and opportunities requires accurate analysis of marketing performance. Customer Data Integration increases the ability of businesses to measure and analyze marketing effectiveness and optimizes resource allocation. To provide more accurate analysis of marketing performance a Customer Data Integration solution that provides for:

  • The integration and aggregation of data to support dynamic metrics tied to specific business strategies and goals
  • The creation of customized, in-memory, multi-dimensional customer data aggregations for rapid analysis and feedback
  • The creation of custom workflows to integrate marketing cost and customer revenue data across lines-of-business

Customer Data Management - Part 4 - Five Key Activities

Customer data management (CDM) is critical to maximizing the return on investment from your customer

Customer data management (CDM) is critical to maximizing the return on investment from your customer relationship management (CRM) system. A primary reason companies fail to maximize their return on CRM investment is data quality problems caused by poor customer data management. If your CDM processes do not provide quality data, the information available in your CRM systems will, at best, be of little use to sales, marketing, customer service, or others. Part 1 of this series discussed the basics of customer data management. Part 2 discussed the need to profile customer data in order to ensure it is complete, accurate, and timely. Part 3 discussed the application of rules to customer data to help manage customer profiles.

This post discusses five key activities that will help ensure customer data management adds value through constant enrichment of customer data. Enrichment is an ongoing process from the time customer data is captured until it is needed to accomplish some business goal. As a general rule, the later a data error is caught in this ongoing value chain, the more expensive and time consuming it is to fix. Moreover, the enrichment of customer data that was flawed from the time of capture is a huge waste of time and resources. Whether you are implementing a new CRM system or seeking to better leverage your existing system, these five activities should serve as a helpful guide.

1. Rank Sources of Customer Data

Rank data profiles from best to worst in terms of quality. If you followed the advice from Part 2, you will have created profiles of various sources of customer data. These sources will have different levels of quality. For example, customer data taken from sales contacts and sales leads is often of lesser quality than customer data used to mail statements or invoices. Data sources with the highest quality can serve as reference data for lower quality sources.

2. Determine Mandatory, Optional, and User Defined Customer Data Fields

For some business needs, it may be necessary to have a physical address, business address, and mailing address. Likewise, telephone numbers can include personal, business, or mobile. Optional, user defined fields may be needed to meet certain business needs. In general, to help determine what data is mandatory, consider the following:

  • What customer data fields are required to meet your business goals?
  • What customer data fields are optional or may be of use in the future?
  • Which user defined customer data fields are mandatory to achieve your business goals?
  • At what stage in the enrichment process must all mandatory fields be available?
  • If a field is designated mandatory, can your primary data profiles consistently and accurately provide the required data?

3. Establish Consistent Formatting Rules

Whether designated persons or an automated system is used to enter data, establish and apply clear and consistent rules for the following:

  • Date formats
  • Address formats - including international addresses, if required
  • Telephone number formats - including international, if required
  • Use and format of acronyms and abbreviations, for example CPA, C.P.A., or Certified Public Accountant
  • The consistent use upper case, lower case, or proper case for name and address data

4. Cleanse Customer Data

Data cleansing can be time consuming and laborious. Consider using an automated tool in conjunction with a data service such as Data.com. At a minimum, the following data cleansing tasks should be performed:

  • Identify and eliminate duplicate records. Based upon available data, formulate and apply rules to identify and eliminate duplicate customer data
  • Spell check mandatory fields such as city, state, country, street, first, and last name. If using proper case, check that the capitalization within a name, such as McLaughlin, is correct.
  • Verify that the address you are using is current
  • Verify that email addresses and telephone numbers are current

5. Establish Procedures to Keep Customer Data Current

Among other things, customers move, get married, change names, change jobs, or retire. On average, about one-third of your customer data becomes obsolete within a one-year period. It is therefore necessary to establish ongoing procedures to ensure that your customer data remains current. Consider performing the following tasks to keep customer data current:

  • Be sure to attach a date created field to each customer record
  • Establish a period of time after the date a record is created, that it must be revalidated. This should be no more than every six months
  • The frequency with which customer data is updated will impact how often it must be revalidated
  • Flag customer data where contact information is dependent upon the time of year. For example, retirees who spend the winter in Florida
  • Establish a period of time after which inactive customer data is archived

Data Integration - A Strategic Imperative - Part One

Over the past several years, an ever growing number of new technology platforms, applications and so

Over the past several years, an ever growing number of new technology platforms, applications and social media channels have continued to emerge. Moreover, a growing number of organizations consider these technologies a critical resource necessary to achieve strategic advantage. However, the full value of that resource must be realized through extraction of information and insights buried in mountains of data.

The Challenge of Leveraging New Sources of Strategic Information

Since all organizations have equal access to the same technology, competitive advantage will be gained by those organizations that overcome the challenge to more efficiently and effectively extract this buried information. Meeting this important challenges requires recognizing that integration of data from numerous, heterogeneous sources has become a critical business function.

Important Questions for Executives and Managers

The recognition of this new, critical role for data integration raises important questions for executives and managers. What data is needed? What are the most valuable sources of critical data; and, most important, are current data integration methods adequate to extract this data? These are challenging questions and there are no easy answers.The Selection of Data Integration Models to Meet Strategic Goals

However, there are answers and that is good news. Further, every organization has the resources required to meet these challenges. Some organizations will certainly find this easier than others. Those organizations that find it easier are likely to already understand or employ strategic data integration. Those that find it difficult may approach data integration as a purely tactical technology or have no approach at all. If your organization is the latter, this series of posts will provide you with some guidelines for shifting from a tactical to strategic data integration approach.  
The First in a Series on Strategic Data Integration

This is the first in a series of posts, the goal of which is to help executives, managers, and technical personnel as they move from tactical approaches to data integration towards a more strategic approach. The guidelines presented in later posts are intended to aid in the evaluation, selection, and application of data integration models and technologies that are required to use information as a strategic asset.

Part Two in this series of posts discusses moving beyond tactical approaches to data integration, the need for a new approach to strategic data integration, and the need to understand fundamental data integration models.

Data Integration - A Strategic Imperative - Part Three

Part Two in this series on strategic data integration discussed the need to move beyond tactica

Part Two in this series on strategic data integration discussed the need to move beyond tactical approaches to data integration. It also discussed the need for a new approach to data integration and the need to understand factors that contribute to the success of strategic data integration. This post continues that discussion and considers the need to avoid complexity and project overreach as factors that contribute to the success of strategic data integration.

Data Integration Complexity

The example in my prior post demonstrates that merging data and functions from disparate applications creates the need to reconcile fundamentally different functions and processes. Attempts to merge fundamentally different systems and applications may therefore result in unnecessary complexity.

From my experience working with financial services firms, and investment management firms in particular, one illustrative example is the integration of data between investment accounting systems and front office analytical or portfolio management applications.

Complexity arises when the goal of a project is to merge or replace the features and functions of one system with those of another. This is often done to save costs by avoiding the need to refactor features and functions. This type of project creates unnecessary dependencies between independent systems and applications.

Complexity is also the result of ignoring another cause of data integration project failure: overreach.

Overreach

One of the most important success factors to understand is project overreach. Failure to avoid project overreach is a major cause of data integration project failure. Studies show that far-reaching, broadly scoped data integration projects have a failure rate ranging from seventy to ninety percent. Generally, overreach manifests in two primary ways:

Large Projects

This cause of failure is best explained by way of an example that is all too prevalent. That is, the pursuit of an end-to-end, all-encompassing data integration solution. The goal of end-to-end data integration projects may seem reasonable. Generally, these projects involve designing and implementing a data integration solution to meet the needs of various users across several lines of business. The goal is to gain efficiency and reduce cost through reuse of data and processes. However, large projects nearly always necessitate ignoring integration constraints.

Scope Creep

Another type of overreach is scope creep. Often, a project begins with limited, well-defined goals that meet a particular business need. However, as a result of interested parties adding their requirements to an ever growing list, the project scope continues to grow. Every integration touch point added results in an exponential increase in complexity. This is often manageable with just a few touch points. However, projects suffering scope creep eventually reach an inflection point that causes the project to simply collapse under its own weight.

Summary of Integration Constraints, Complexity, and Overreach

Integration constraints, complexity, and overreach are not necessarily the cause of project failure. Generally, they are a symptom of a broader failure.

How can overreach be avoided? Which data should be included in an integration project and which excluded? How can firms execute strategic data integration projects that meet business needs, stay within scope, and provide value over the long-term?

These are fundamental questions and answering them first requires an understanding basic data integration models. Beyond this understanding, it is important to specify business goals and identify the data integration model that is best used to achieve that goal.

Large Data Integration Projects

This cause of failure is best explained by way of example. One example, which occurs all too often, is the pursuit of an end-to-end, all-encompassing data integration solution. The goal of end-to-end data integration projects may seem reasonable. Generally, these projects involve designing and implementing a data integration solution to meet the needs of various users across several lines of business. The goal is to gain efficiency and reduce cost through reuse of data and processes.

Data Integration Project Scope Creep

Another type of overreach is scope creep. Often, a project begins with limited, well-defined goals to meet a particular business need. However, as a result of interested parties adding their requirements to an ever growing list, the project grows in scope and complexity. With each new integration touch point added, the complexity of the project grows exponentially. Finally, the project simply collapses under its own weight.

Summary - Integration Constraints, Complexity, and Overreach

Constraints, complexity, and overreach are not necessarily the cause of project failure. Generally, they are a symptom of a broader failure.

How can overreach be avoided? Which data should be included in an integration project and which excluded? How can firms execute strategic data integration projects that meet business needs, stay within scope, and provide value over the long-term?

These are fundamental questions and answering them first requires an understanding basic data integration models. Beyond this understanding, it is important to specify business goals and identify the data integration model that is best used to achieve that goal.

Part Four in this series discusses the role data integration model selection plays in the success of strategic data integration initiatives.