Economics of Software Resiliency

Resilience is a design feature that facilitates the software to recover from occurrence of an disruptive event. As it is evident, this is kind of automated recovery from disastrous events after occurrence of such events. Yes, given an option, we would want the software that we build or buy has the resilience within it. Obviously, the resilience comes with a cost and the economies of benefit should be seen before deciding on what level of resilience is required. There is a need to balance the cost and effectiveness of the recovery or resilience capabilities against the events that cause disruption or downtime. These costs may be reduced or rather optimized if the expectation of failure or compromise is lowered through preventative measures, deterrence, or avoidance.

There is a trade-off between protective measures and investments in survivability, i.e., the cost of preventing the event versus recovering from the event. Another key factor that influences this decision is that cost of such event if it occurs. This suggests that a number of combinations need to be evaluated, depending on the resiliency of the primary systems, the criticality of the application, and the options as to backup systems and facilities.

This analysis in a sense will be identical to the risk management process. The following elements form part of this process:

Identify problems

The events that could lead to failure of the software are numerous. Developers know that exception handling is an important best practices one should adhere to while designing and developing a software system. Most modern programming languages provide support for catching and handling of exceptions.  This will at a low level help in identifying the exceptions encountered by a particular application component in the run-time. There may be certain events, which can not be handled from within the component, which require an external component to monitor and handle the same. Leave alone the exception handling ability of the programming language, the architects designing the system shall identify and document such exceptions and accordingly design a solution to get over such exception, so that the system becomes more resilient and reliable. The following would primarily bring out possible problems or exceptions that need to be handled to make the system more resilient:

• Dependency on Hardware / Software resources - Whenever the designed system need to access a hardware resource, for example a specified folder in the local disk drive, expect a situation of the folder not being there, the application context doesn't have enough permissions to perform its actions, disk space being exhausted, etc. This equally applies to software resources like, an operating system, a third party software component, etc.
• Dependency on external Devices / Servers / Services / Protocols - Access to external devices like printers, scanners, etc., or other services exposed for use by the application system, like an SMTP service for sending emails, database access, a web service over HTTPS protocol, etc. could also cause problems, like the remote device not being reachable, or a protocol mismatch, request or response data inconsistency, access permissions etc. 
• Data inconsistency - In complex application systems, certain scenarios could lead to a situation of inconsistent internal data which may lead to the application getting into a dead-lock or never ending loop. Such a situation may have cascading effect as such components will consume considerable system resources quickly and leading to a total system crash. This is a typical situation in web applications as each external request is executed in separate threads and when each such thread get into a 'hung' state, over a period, the request queue will soon surpass the installed capacity. 

Cost of Prevention / recovery

The cost of prevention depends on the available solutions to overcome or handle such exceptions. For instance, if the issue is about the SMTP service being unavailable, then the solution could be to have an alternate redundant, always active SMTP service running out of a totally different network environment, so that the system can switch over to such alternate service if it encounters issues with the primary one. While the cost of implementing the handling of multiple SMTP services and a fail-over algorithm may not be significant, but maintaining redundant SMTP service could have significant cost impact. Thus with respect to each such event that may have an impact on the software resilience, the total cost for a pro-active solution vis-a-vis a reactive solution should be assessed.

Time to Recover & Impact of Event

While the cost of prevention / recovery as assessed above will be an indicator of how expensive the solution is, the Time to Recover and the Impact of such an event happening will indicate the cost of not having the event handled or worked around. Simple issues like a database dead-lock may be reactively handled by the DBAs who will be monitoring for such issues and will act immediately when such an event arise. But issues like, the network link to an external service failing, may mean an extended system unavailability and thus impacting the business. So, it is critical to assess the time to recover and the impact that such an event may have, if not handled instantly.

Depending on the above metric, the software architect may suggest an cost-effective solution to handle each such events. The level of resiliency that is appropriate for an organization depends on how critical the system in question is for the business, and the impact of the lack of resilience for the business. The organization understands that the resiliency has its own cost-benefit. The architects should have this in mind and design solutions to suit the specific organization.

The following are some of the best practices that the architects and the developers should follow while designing and building the software systems:

• Avoid usage of proprietary protocols and software that makes migration or graceful degradation very difficult.
• Identify and handle single points of failure. Of course, building redundancy has cost.
• Loosely couple the service integrations, so that inter-dependence of services is managed appropriately.
• Identify and overcome weak architecture / designs within the software modules or components.
• Anticipate failure of every function and design for fall-back-scenarios, graceful degradation when appropriate.
• Design to protect state in multi‐threaded and distributed execution environments.
• Expect exceptions and implement safe use of inheritance and polymorphism 
• Manage and handle the bounds of various software and hardware resources.
• Manage allocated resources by using it only when needed.
• Be aware of timeouts of various services and protocols and handle it appropriately

Governance of Agile Delivery

Introduction

The Agile methodology brings in alternate approach to traditional project management, where success was hard to get. Typically used in software development, Agile methodology help businesses respond to unpredictability. By focusing on the repetition of smaller work cycles as well as the deliverables, agile methodology is described as “iterative” and “incremental”. In waterfall, development teams only have one chance to get each aspect of a project right. In an agile paradigm, every aspect of development viz. requirements, design, etc. is continually revisited. When a team stops and re-evaluates the direction of a project every two weeks, there’s time to change course. Because teams can develop software at the same time they’re gathering requirements, “analysis paralysis” is less likely to impede a team from making progress. Agile development preserves a product’s critical market relevance and ensures a team’s work doesn’t wind up on a shelf, never released. Considering the value delivery that the Agile methodology promises, its adoption has been on the rise and today most organizations, including Government are embracing Agile approaches.


Governance of Agile Delivery


Critics say that Agile methodology is all about working in an unstructured way and for that reason, they believe that governing agile practices is always a challenge. While some of the Agile principles appear to support such criticism, there are many cases where organizations have successfully implemented processes and frameworks towards governance of Agile practices. Agile practitioners believe that because the agile methods are designed to be self-assuring, when practiced right, there exists built-in governance and accountability.


More so, the agile practices are more collaborative and operates continuously, requiring the stakeholders to review and test the deliverables on a continuous basis and helps the team to take alternate course of action as may be needed. Collaborative culture helps resolution of problems quicker and makes decisions are made on time. This helps to have a continuous focus on the value forecast with respect to the business case and manage the risks that may potentially impact on the expected value.


Principles of Governance

The following are the key governance principles for a successful governance of Agile Delivery:

Focus on the value delivery - only do a task if it brings value to the business. This principle also recognizes the timely delivery of a task as the value derived is more likely to deteriorate with the delayed delivery. In case of Agile deliveries, the governance is continuous and at a work unit level. It should also focus on what activity is taking place and the value such task delivers.

Embrace Change - This another principle of Agile and the Governance framework should take this into consideration. This would mean that the decisions or work flows should be flexible enough to change course based on the feedback received. Given that all stakeholders collaborate, decisions should be taken across the table, without putting things on hold and for the purpose, all needed specialists should take part in the reviews.

Decide on the performance metrics - Another key principle of Agile methodology is to 'fail fast and learn quiuckly'. Given that the overall objective is to improve the certainty that the team will deliver a usable product or service of good quality, the teams should be able to identify and implement the right metrics that will accurately indicate the quality of the deliverables and the performance of the team. For example they measure tasks completed; rework they had to perform; the backlog list and the value of the product or service to the business at the end of each iteration. Teams display this information visually, updating it frequently. This makes progress transparent to business users and management. If senior managers require performance information to oversee projects, they define what the ‘must have’ data are. Performance reports for senior management become a task in each iteration and an output of the delivery team.

Collaboration - All stakeholders, including senior management, external assessors, business users and the development team should be partners in quality, and this collaborative approach is an essential change in mindset. The business owner and delivery team defines what ‘quality’ tests they will use and what results are acceptable at the outset of each iteration – the definition of ‘done’. Regular user feedback identifies whether the product or service is providing the expected business value at each stage. External assessors are not gatekeepers; rather they are an integral part of the team. The iterative approach ensures continual reviews and feedback on progress, so external assessors are not just involved at critical points as defined in a traditional project life cycle.

Focus on behaviours and not just processes and documentation - More specifically, the external reviews or assessments will be more effective in providing critical challenge if the assessors have high-end skills, including technical and Agile delivery experience. In addition, they provide better value if they continually review how the team is performing, using observation as their main method of evidence collection. The focus of such external review or assessment shall be on the following:

• the skills and experience of the team;
• the team dynamics – frequency and nature of communication inside and outside of the delivery team, and the level of input to the delivery team from the business;
• the organisational culture – the level of commitment and openness;
• the timing and nature of quality control by the delivery team – the testing and release framework;
• the order in which the team tackled the tasks – prioritisation of actions and deliverables, the amount of actions in the backlog list;
• the way the team changes its activity in response to the results achieved in each iteration; and
• the value of outputs to the business.

IBM's Disciplined Agile Delivery Methodology


IBM believes Agile delivery allows it to continually issue new capabilities that meet user needs. It usually introduces software as part of a wider business change project so, to keep both in step, it has developed several Agile project methodologies. Disciplined Agile Delivery is a hybrid method that can be applied by a large number of teams working on the same project at the same time. The image below shows the Disciplined Agile Delivery life cycle. It starts with a few short iterations that allow the team and its stakeholders to identify the initial requirements, develop the architecture and agree a release plan. IBM also uses this to determine the system level properties and characteristics – the non-functional requirements. There are iterations after the business owner has decided that the system has sufficient functionality. These additional iterations are necessary for IBM to support the operation and maintenance of the solution once it is in service.



In contrast to the traditional approach of looking at outputs, plans, resourcing and how a project is organised, external assessors should focus on outcomes, prioritisation of work and team dynamics. The most useful indicators of success are how the teams are organising the delivery of an operational service or capability and what Agile behaviours and practices are used. Areas for assessment include whether:
system level issues (security, availability) are addressed within the iterations;

• short- and longer-term planning exists;
• the stakeholders have a shared vision;
• there is continuous integration; and
• the team has the right people

---

Reference:

National Audit Office's Review on Governance of Agile Delivery

IT Governance - Implementation Obstacles

IT governance is a process which include a set of controls and practices that ensures that the IT function is working on the right things at the right time in the right way with a view to accomplish the stated objectives and thereby contributing towards the meeting enterprise objectives and goals. Any process that aligns IT to business goals is the right strategy. However, it’s the change required and the compromises on the part of business leaders that can come in way to make it a not so easy program.

IT Governance offers many benefits, which include reduce the cost of day-to-day operations, improve overall operational efficiency and consistency, free more resources for strategic initiatives that improve competitiveness, choose those initiatives far more wisely working on the right things, bring those initiatives to market faster with less risk and bring IT into close alignment with business priorities. But at the same time the results of an ineffective implementation can be devastating. Some such devastating results could be:

• Business losses and disruptions, damaged reputations and weakened competitive positions
• Schedules not met, higher costs, poorer quality, unsatisfied customers
• Core business processes are negatively impacted (e.g. SAP impacts many critical business processes) by poor quality of IT deliverables 
• Failure of IT to demonstrate its investment benefits or value propositions

The Three Pillars of IT Governance

To understand the obstacles to IT Governance in an organization, it would be appropriate to understand the three critical pillars on which a successful IT Governance program is built on. The following are the three critical pillars of a successful IT Governance implementation:

Leadership, Organization, Decision Rights and Metrics

The IT Governance Initiative must be decomposed into manageable and accountable work packages and deliverables and assigned to owners for planning, development, execution and continuous improvement. The IT Governance program must have clearly defined roles, responsibilities and decision rights for the entire program and for each major component of the integrated IT Governance framework and road map.
A decisions rights matrix identifying decision influencers and decision makers is necessary to clarify decision roles and authority levels for the major IT Governance components.

Flexible and Scalable Processes

Processes form an integral part of the IT Governance program and as the IT Governance framework is made of such processes and controls, which shall be defined. It is also important these processes evolve over its usage based on feedback collected through various metrics. At the same time, processes should not only be simple enough to understand and implement but also flexible enough to provide room for improvement. People tend to ignore processes, if it is difficult to understand and practice as part of their day to day work. Thus the integrated framework approach works best.

Enabling Technology

Most business components rely on Technology for most aspect of their value, reliability or efficiency. Even choice of right technology plays a key role in making up the first two pillars. Given that technology evolves in an accelerated rate, there should be a clear watch on such advancements and the technology road map should provide for identification and adoption of the right technology at the right time to get the maximum value. Most organizations have recognized and accordingly have started managing this area well.

The Key Obstacles

Most often, the business leaders are motivated and rewarded by having their small part of the organization succeed. IT governance requires that the scarce resource of technology capacity be diligently distributed across the organization for overall business success. In other words, it requires that IT cannot be allocated on the basis of individual team needs but rather on collective, organizational goals. A recent empirical study by Lee uncovered factors such as ‘lack of IT principles and policies’, ‘lack of clear IT Governance processes’, ‘lack of communication’, and ‘inadequate stakeholder involvement’, as inhibitors of IT Governance implementation success. A good understanding on the barriers or obstacles that hinder the success of IT Governance implementation is important as once understood, their effect is understood and pre-emptive actions can be taken to address them

Implementing IT Governance is a long and continuous journey, where obstacles and challenges are aplenty. A good understanding on the barriers or obstacles that hinder the success of IT Governance implementation is important as once understood, their effect is understood and pre-emptive actions can be taken to address them. The most frequently experienced obstacles include:

Culture

Instituting effective IT governance requires dealing with the “c-word.” The culture of a company—“the way we do things here”—can be a tremendous driver for business success. It can also be—and often is—a giant resistor that dampens positive change. Immeasurable amounts of energy have been dissipated trying to change embedded habits and methods that hid behind the cloak of “culture.” Today, worldwide, the trend is toward collaborative culture, especially in the sharing of information. The attitude that “information is power” lingers in some dark company corners. In some disciplines, such as sales, where compensation is directly related to personal contacts and initiative, it is arguable that the status quo has value. In most cases, though, managements are trying to rid the company of these attitudes in order to unlock the power of teamwork leveraged by technology. IT governance requires teamwork and information sharing to succeed.

Resistance to Change

Virtually every manager in business today has encountered employees who held up organizational change by insisting on continuing with the “old way” of doing something, even though the success of the “new way” depends on universal adoption. Fear of failure could be one of the reason why people are afraid to commit to change, uncertain that they can successfully implement it and fearing that if they fail, they will be held accountable. Another reason could be the existence of innate conservatism and uncertainty emanating and causing resistance

Lack of Appropriate Communication

Communication is really at the heart of IT governance and the lack of appropriate communications can cause a major disconnect between IT executives and business executives. IT still continues to communicate in more technology terms, which is just not relevant to the business and they just don't understand it. So good communications is extraordinarily important so that everybody is on the same page and that the business and IT become very closely engaged. Again -- we're making strategic decisions on where we're going to invest in technology and those are really business decisions, not technology decisions. That way, lack of communication can easily derail the IT Governance program of an organization.

Lack of Value Proposition

CIOs must be willing to take the lead in the search for value-creating IT processes. If they are not, others—real experts—are glad to do so, in language that resonates with CEOs. For instance, if you take the Project and Porfolio Governance the 'Fail Fast' or 'Fail First' approach may be helpful. If the processes are designed around this approach, we could see that the IT programs and functions get evaluated at various stages by analyzing the collected metrics to see if it would still make sense to let the project, or program to move into the next stage. At every stage there using the metrics, a revisit to the project charter and the business objectives would ensure that the desired value out of such project or program is still the same.

Internal politics

Internal organizational politics may exert themselves, as the adoption and implementation of formal ITG practice will sometimes bring a shift in decision rights and associated powers that currently exist in the organization. It is seen in most organizations that projects that should be given a higher priority mostly be based on “who speaks the loudest” rather than“ looking at the current business, collected metrics, what is the immediate need?”

Effective vs Ineffective Security Governance

Continuing with my earlier blog on Measuring the Performance of EA, I was looking for methods and measures that can be used for measuring the effectiveness of the security program in an enterprise. I happened to read a CERT article titled as Characteristics of Effective Security Governance which contains a good comparision of what is effective and what is ineffective. I have reproduced it here in this blog for a quick reference. The original article of CERT though out dated is worth reading.

Effective	Ineffective or Absent
Board members understand that information security is critical to the organization and demand to be updated quarterly on security performance and breaches. The board establishes a board risk committee (BRC) that understands security’s role in achieving compliance with applicable laws and regulations, and in mitigating organization risk. The BRC conducts regular reviews of the ESP. The board’s audit committee (BAC) ensures that annual internal and external audits of the security program are conducted and reported.	Board members do not understand that information security is in their realm of responsibility, and focus solely on corporate governance and profits. Security is addressed adhoc, if at all. Reviews are conducted following a major incident, if at all. The BAC defers to internal and external auditors on the need for reviews. There is no audit plan to guide this selection.
The BRC and executive management team set an acceptable risk level. This is based on comprehensive and periodic risk assessments that take into account reasonably foreseeable internal and external security risks and magnitude of harm. The resulting risk management plan is aligned with the entity’s strategic goals, forming the basis for the company's security policies and program.	The CISO locates boilerplate security policies, inserts the organization's name, and has the CEO sign them. If a documented security plan exists, it does not map to the organization’s risk management or strategic plan, and does not capture security requirements for systems and other digital assets.
A cross-organizational security team comprised of senior management, general counsel, CFO, CIO, CSO and/or CRO, CPO, HR, internal communication/public relations, and procurement personnel meet regularly to discuss the effectiveness of the security program, new issues, and to coordinate the resolution of problems.	CEO, CFO, general counsel, HR, procurement personnel, and business unit managers view information security as the responsibility of the CIO, CISO, and IT department and do not get involved. The CSO handles physical and personnel security and rarely interacts with the CISO. The general counsel rarely communicates particular compliance requirements or contractual security provisions to managers and technical staff, or communicates on an ad-hoc basis.
The CSO/CRO reports to the COO or CEO of the organization with a clear delineation of responsibilities and rights separate from the CIO. Operational policies and procedures enforce segregation of duties (SOD) and provide checks and balances and audit trails against abuses.	The CISO reports to the CIO. The CISO is responsible for all activities associated with system and information ownership. The CRO does not interact with the CISO or consider security to be a key risk for the organization.
Risks (including security) inherent at critical steps and decision points throughout business processes are documented and regularly reviewed. Executive management holds business leaders responsible for carrying out risk management activities (including security) for their specific business units. Business leaders accept the risks for their systems and authorize or deny their operation.	All security activity takes place within the security department, thus security works within a silo and is not integrated throughout the organization. Business leaders are not aware of the risks associated with their systems or take no responsibility for their security.
Critical systems and digital assets are documented and have designated owners and defined security requirements.	Systems and digital assets are not documented and not analyzed for potential security risks that can affect operations, productivity, and profitability. System and asset ownership are not clearly established.
There are documented policies and procedures for change management at both the operational and technical levels, with appropriate segregation of duties. There is zero tolerance6 for unauthorized changes with identified consequences if these are intentional.	The change management process is absent or ineffective. It is not documented or controlled. The CIO (instead of the CISO) ensures that all necessary changes are made to security controls. In effect, SOD is absent.
Employees are held accountable for complying with security policies and procedures. This includes reporting any malicious security breaches, intentional compromises, or suspected internal violations of policies and procedures.	Policies and procedures are developed but no enforcement or accountability practices are envisioned or deployed. Monitoring of employees and checks on controls are not routinely performed.
The ESP implements sound, proven security practices and standards necessary to support business operations.	No or minimal security standards and sound practices are implemented. Using these is not viewed as a business imperative.
Security products, tools, managed services, and consultants are purchased and deployed in a consistent and informed manner, using an established, documented process. They are periodically reviewed to ensure they continue to meet security requirements and are cost effective.	Security products, tools, managed services, and consultants are purchased and deployed without any real research or performance metrics to be able to determine their ROI or effectiveness. The organization has a false sense of security because it is using products, tools, managed services, and consultants.
The organization reviews its enterprise security program, security processes, and security’s role in business processes. The goal of the ESP is continuous improvement.	The organization does not have an enterprise security program and does not analyze its security processes for improvement. The organization addresses security in an ad-hoc fashion, responding to the latest threat or attack, often repeating the same mistakes.
Independent audits are conducted by the BAC. Independent reviews are conducted by the BRC. Results are discussed with leaders and the Board. Corrective actions are taken in a timely manner, and reviewed.	Audits and reviews are conducted after major security incidents, if at all.

The article also lists eleven characteristics of effective security governance in addition to listing the Ten challenges to implementing an effective security governance. I would highly recommend you to read the full article.

---

References:
CERT’s resources on Governing for Enterprise Security

CERT and CERT Coordination Center are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

---

Implementing IT Balanced Scorecard

Source: ISACA, Board Briefing on IT Governance 2nd edition

With IT increasingly becoming an enabler of business, more and more organizations are looking for effective and efficient management of IT, so that the investment in IT fetches optimum value. On the same lines, the need for better IT Governance is being felt by the Board of increasing number of organizations. One of the key domain of IT Governance is Performance Measurement. Going by "what is not measured cannot be managed", there need to be plans and processes in place for measuring the performance of IT so that it can be better governed.

Much of the value returned by IT are intangible. While it is easy to measure the tangible benefits, measuring intangible benefits is difficult. Business Scorecard (BSC) which evolved in the early 1990s has evolved into an very useful tool for measuring both tangible and intangible benefits segmented into four perspectives - Financial, Customer, Internal Process and Learning. IT BSC as derived from the Business Scorecards were found to be a a very effective measurement system addressing the concerns of reporting the intangible benefits to the Board.
The Balanced Scorecard as it has evolved over a period of time is being looked at not just as a performance measurement tool, but as a strategic planning and management system. This is because, the Balanced Scorecards can be cascaded down smaller business units including IT and aggregated upwards to the higher-level. IT BSC, which is cascaded from the Business Scorecard can be further subdivided into one for each of the technology domains, for instance one for managing the IT Operations and another to manage the IT Development areas. While doing so, it is important to maintain the linkages between each such cascaded Scorecards and this way the Balanced Scorecard can facilitate Strategy Mapping, thereby improving the Alignment of the objectives of the smaller business and IT units into the business strategy.


The perspective of the IT BSC may be redefined to better represent the IT organization. For instance, the following four perspectives may be used in IT BSC:

• Corporate Contribution - Equivalent to the Finance perspective of the Balanced Scorecard, this represents the view of business executives on the IT department. 
• Customer Orientation - Equivalent to the Customer perspective of the Balanced Scorecard, this represents the view of the end users on the IT department. 
• Operational Excellence - Equivalent to Process perspective of the Balanced Scorecard, this represents the effectiveness and efficiency of various standards, processes and policies followed by the IT department. 
• Future Orientation - Equivalent to Learning and Growth perspective of the Balanced scorecard, this represent a view of how well IT is prepared to meet the future needs of the business.

To be effective, the following three principles need to be built into the balanced scorecards:

• Cause-and-effect relationships - the identified performance measures have a cause and effect relationships amongst them, for instance a measure on Improved developer skills (Future Orientation perspective) as a cause will result in improved quality in the applications delivered(Operational Excellence perspective), which in turn should contribute for user statisfaction (User Orientation perspective) 
• Sufficient performance drivers - While it is common to measure all the possible outcomes (measuring what you have done), it is also important to identify and include suufficient performance drivers(how you are doing). A good mix of both outcome measures and performance drivers are essential for the Scorecard to be effective. 
• Linkage to financial measures - IT Scorecard, being cascaded from the Enterprise Business Scorecard, the measures in the IT Scorecard should link up to a corresponding measure in the top-level business scorecard. 

To have the Balanced Scorecard implemented as part of the IT Governance initiative, the following steps are recommended:

• Obtain commitment - Make a presentation to the board and executives explaining the concepts, benefits and cost of implementing it and get a commitment to go ahead. 
• Kick-off - Kick off the Balanced Scorecard initiative as a project and as part of this activity, train the staff and identify the project team members. 
• Strategy map - Get an understanding of the corporate business strategy and the sub unit level strategies and then establish a strategy map. 
• Metrics selection - Understand the existing metrics if any and identify the required metrics, which should be a good mix of both outcome measures and performance drivers 
• Metrics definition - With respect to each identified metric, create a standard definition, related processes to collect and manage the data. As part of this, the cause and effect relationships should also be clarified and the linkage with higher level scorecards should also be established. 
• Assign ownership - Assign owners for each metric. 
• Define Targets - With respect to each metric, set targets (may be a range) for the function heads to achieve and devise strategic initiatives to achieve these targets. 
• Act on the results - Have the appropriate executive management or board as may be required to review the resulting measures and then act on the results. 
• Review periodically - The metric definitions, the linkages and the cause-effect relationships may require revision based on experience and this achieved through periodic reviews. 

Successful execution of strategy requires the successful alignment of four components: the strategy, the organization, the employees and the management systems. As Kaplan and Norton put it, “Strategy execution is not a matter of luck. It is the result of conscious attention, combining both leadership and management processes to describe and measure the strategy, to align internal and external organizational units with the strategy, to align employees with the strategy through intrinsic and extrinsic motivation and targeted competency development programs and finally, to align existing management processes, reports and review meetings, with the execution, monitoring and adapting of the strategy.”

Measuring Performance of Enterprise Architecture

Enterprise Architecture is viewed as an important function in IT Governance and it plays a vital role in aligning IT with the Business. This function is expected to define the technical direction and to ensure application of principles of Architecture to the design and maintenance of IT systems, which in turn should be in alignment with the business vision, mission and strategies and the role that the IT within the organization is expected to play. While the support for commitment and funding is important, it is also important that the EA function consider the following(not exhaustive) to be successful:

• Alignment with the business strategy and the culture of the organization, 
• Actively involve in projects to ensure that the principles of design and evolution are adhered to and ensure the continued focus on the business requirements.
• Offer technical consultancy for all the business and IT functions both internal and external. 
• Acting as a gate for all decisions impacting the design & evolution architecture. 

IT Governance views EA as the hub of the IT wheel with linkages with various processes, components and goals of the enterprise and some of such key and enabling links are:

• Promoting and enabling Business Agility
• Providing standards, policies and principles for the IT Project, Program and Portfolio management function
• Guides and enables cost management and consolidation
• Facilitates cost-effective, scalable integration of various IT systems
• Supports IT Governance by defining / providing the conceptual and technical priorities and thereby promoting informed decision making

Going by the premise, what is not measured does not get managed, it is important to identify the measurable objectives for the Architecture function itself, so that it is well managed and its contribution to the success of the organization is established. While measurement of Architectural activities is difficult, COBIT suggests a set of measurable outcomes and performance measures, some of which are the following:

Number of technology solutions that are not aligned with the business strategy - One of the objectives of the EA function should be to ensure that the technology solutions chosen or implemented are aligned to the business strategy a measure around this could be very useful to establish that the number of misaligned solutions are on the decline. There could be other measures derived around this and could be represented as a relative measure to the total solutions.

Percent of non-compliant technology projects and platforms planned - With the fast changing business environment, there will be times when the business will need to solutions technology projects that are not compliant with the standards and principles laid out by EA. EA has the responsibility to carefully review such needs and grant waivers. Such waivers should be for a shorter term and should be backed with a plan to normalize it. At times, this could call revision in the standards, policies or principles. A measure around this could be very useful that the EA is effective in dealing with non-compliant technology projects and platforms.

Decreased number of technology platforms to maintain - Standardization is one of the objectives of EA, which could contribute to cost reductions and reduce the technical complexity. Statistics and surveys show that enterprises without an active IT Governance / EA function have more multiple applications requiring different platforms for the same business requirement being used by different departments. With an effective EA function, these should be very less in number and should decline over a period. A measure around this is a very good indicator of EA being effective in this area.

Reduced application deployment effort and time-to-market - Supporting Business agility is yet another key objective of the EA function. Today’s businesses are operating in a highly dynamic industry environment and in order to stay competitive and to sustain its market position, need support from IT to have the new or changed capabilities with a reduced time-to-market. A delayed delivery from IT could mean an opportunity lost. A measure around this indicator would really be helpful in establishing how IT is supporting the business changes.

Increased interoperability between systems and applications - It is quite common that most enterprises have multiple applications for specific needs, but there is a need to have these applications share data and information amongst each other. With cloud computing gaining wider acceptance, most enterprises are looking at discrete cloud based solutions. With the benefits outweighing the concerns and constraints, and that the industry is working towards addressing these concerns, there will be increased focus on move to cloud. This will mean hosted applications from various providers would need to be interoperable and working with other in house applications. EA should ensure that the technology and solutions acquired or designed should support this important attribute i.e. interoperability. A measure around this parameter would be an important indicator of EA function’s effectiveness.

Percent of IT budget assigned to technology infrastructure and research - Yet another expectation from the EA function is that it should help businesses in leveraging emerging technology to its advantage. This will require the Architects to be continuously looking for newer technologies and its application areas, and recommend such technology or solutions for implementation so that the business will get the most out of IT to accomplish its mission. It is also important the extent of this activity should be in line with the identified and stated role of IT in the organization. While the percentage of IT budget used for research is an useful measure, there could be other useful measures derived from this, for instance, number of research solutions getting implemented as a percentage to total number of solutions implemented in a given period. Business satisfaction on timely identification and analysis of technology opportunities is another related measure which is indicative of the outcome of this research.

Number of months since the last technology infrastructure review - With the fast changing IT space, it is important to ensure that the technology infrastructure is of continued relevance to meeting the business objectives and if needed changes should be considered. A measure indicating that the EA function is performing review of the technology infrastructure periodically is a good indicator of its effectiveness. Measures derived around this review could be based on the outcome of the review will also be very useful.

There is no one size fits all in IT and as such the measures indicated above need to be tailored to suit the organization and the role of IT within the organization.

References:

COBIT - an IT Governance framework from ISACA.

Developing a successful governance strategy - A best practice guide for decision makers in IT from The National Computing Center, UK

---

Pre-project Reviews help projects fail at start

A new project was approved and a 15 member team sit round the table in the project kick off meeting with excitement and enthusiasm. Business Analysts have put together the stories which Developers have started developing. There were periodical project status reviews and there were issues and risks discussed in these meetings and the project manager did well in managing the issues and risks. There comes a message from stakeholders that all work on the project be stopped with immediate effect.

That may be sounding familiar to every IT familiar as study says that 37% of the IT projects are at the risk of failing. Another study finds that 31% of the projects are cancelled before even hitting the finish line.  It really hurts the project team members when a project is cancelled or shelved. But there could be reasons for doing so and such decisions are taken when continuing with the project will only increase the loss and would not bring in value for the sponsors. This is where ‘fail fast’ will help as some times pulling the project down even earlier would save a lot of efforts and money for the sponsors.

Let us see what can be done even before starting any work on a project, so that the potential failure is spotted in the pre-project stage itself so that the project is not allowed to start in the first place. When the need for a project is felt, the executive management (sometimes called Project Review Board) takes a look at it and reviews it and if it sees merit in it, gives a go for the project. In some cases, by the time the Project Review Board (PRB) sees the project proposal, considerable efforts would have already incurred in the form of requirement gathering and analysis. The review by the PRB members, when done well will bring out the ability to measure the risk exposure and the RoI (Return on Investment) of the project, which in turn influences the further decision. However, it is important here that the PRB members shall be presented with adequate information that helps taking the right decision. A well drafted checklist or template that captures all the required information would help not to miss out the details and will help reduce the projects failing down the road.

The project proposal template shall at a minimum capture the following details, so that an informed decision is taken by the PRB.

Motivation:

The problem the project is expected to solve or a business opportunity that the project will help the business to capitalize should be stated well, so that the PRB members are able to appreciate the motivation behind the project need. It would be appropriate to quantify the value / benefit the solution would bring when implemented as intended. Some projects may bring immediate benefits, whereas some may bring benefits in the longer term, in which case, the same shall be called out explicitly. Similarly some projects may bring in monetary benefits, while others may bring in intangible value / benefits.

Estimated investment:

It may not make sense if the investment to be made for the project is significantly higher, which may leave the value or benefit negligible. Most projects get hit on account of cost overrun. The estimate should be close to being accurate and use of a template will make sure that all cost elements are considered. In some cases, a better estimation can be made only after gathering requirements with sufficient details. It would be wise that in such cases, the PRB may take a call to sponsor the initial efforts alone and let the proposal be placed again for review with more accurate estimation.

Constraints:

All projects will be constrained with various ifs and buts. Each possible constraint should be identified and called out in the proposal document. Each constraint will have one more associated risk items for which a contingency plan and mitigation plan has to be put in place. Risk Management practice has evolved in recent times and there are methods and techniques with which the risks can be quantified (a factor of probability and impact) and the overall risk exposure of the proposed project can be measured. This will help the reviewers to take an informed decision whether this much risk is worth to be taken for the value this project might give back.

Solution longevity and re-usability:

It would help if the expected life of the solution is determined. Some of the solutions may have shorter life, but may offer the re-usability with minor changes / enhancements. This will have to be determined considering the longevity of the tools and technology that forms part of the solution. Considerable number projects get cancelled as the sponsors gets to know that the useful life of the solution is going to be short and that the project may not be completed on time leading to further reduction in the longevity.

The above are some of the key factors that influence the decision on the project. There are many other factors which depending on the nature and size of the project may have significant influence on the decision making. For instance, security and compliance requirements could be a significant risk, which the PRB needs to know of while taking the go decisions on projects. Other factors that may find a place in the Proposal document include, deployment infrastructure, post production maintenance aspects, choice of technology, resource availability, etc.

On top of everything, the team that prepares the project proposal document should have the expertise in the business domain, technology used, estimation and related skill areas. Typically this will require a team of architects of appropriate specialization to accomplish this.

References:

http://www.zdnet.com/blog/projectfailures/cio-analysis-why-37-percent-of-projects-fail/12565

http://www.it-cortex.com/Stat_Failure_Cause.htm

http://www.articlesbase.com/training-articles/project-failure-what-are-the-reasons-for-and-statistics-on-it-871395.html

Governing Identity Management

Traditionally, each software application is developed to maintain and manage the identity and the related permission information within it. As more and more such applications gets deployed, user provisioning and managing access control could soon be a nightmare. A well managed Identity Management function within an enterprise can alleviate the hassles around this and will also enable the enterprise to better govern the identity and resource provisioning activities.

Identity Management solution as such comprises of the following key functions in addition to being technically capable of exposing necessary automation APIs:

Account Provisioning

This is a core function within Identity Management and this is where an identity gets created.  The following are the typical activities that need to be performed under this function.

• Adding an Identity - includes receiving a request with required data, performing necessary verification and obtaining approval from appropriate authority.
• Modifying an Identity - involves change of certain attributes of an identity.
• Deleting an Identity - when an identity no is no longer associated with the organization, deletion may be required. Deletion may not mean actual deletion and instead may mean de-activation.
• Suspending / Resuming an Identity - usually when employees go on long vacation, it would be appropriate to suspend the identity and resume again when the employee comes on board.

Resource Provisioning

An Identity once created need to be provisioned to access one or more services, which could be out of a computing resource or a non computing service. For instance, computing resources could mean access to payroll application and similarly a non computing resource could mean physical access to the Data Center.

De-Provisioning

De-provisioning is an equally important function which, if not done on a timely manner could put the organization into a big risk. For instance, if an employee who has been granted access to critical systems, is not de-provisioned when he leaves the organization, he could cause potential loss to the company.

Managing Permissions and Authorization

Provisioning a resource would only mean that the resource has a need to use the target resource, but it has to be further managed by defining specific privileges like, Read, Write, Delete. Similarly, the identity may have to be granted different permissions for different sub functions that the resource may expose. While a standards based IAM solution would be extensible, the consuming application may require changes to interact with the IAM solution and make use of the authorization information that is exposed.

Governance

With a central identity management solution, it is important that the related functions are better managed, monitored and audited. This requires defining, implementing and monitoring controls around people, process and tools & technology.

• People – The person performing the one or more of the above functions should be highly trust worthy and appropriate separation of duties and responsibilities should be put in place. For instance, the person approving the identity creation should not be the same person who creates it. The identity performing these functions should be at appropriate level which ensures accountability. 
• Process – Policies and processes need to be defined for each of the above functions. For instance, Identity creation shall specify the source of data, the required attributes for which data need to be captured, a process or methodology to have the identity information verified and on top an approval process. It is typical that the approving authority may be different for different resource, which has to be unambiguously defined. There should also be a process specifying the monitoring and audit requirements for the above functions.
• Tools  & Technology – Carrying out the above functions will certainly need an appropriate tool and related technology. A comprehensive enterprise tool may facilitate carrying out all the required functions in addition to offering necessary APIs for the resources that consumes the authentication services. It is important to specify how access to these tools and related infrastructure is protected and governed.

The following are the key control objectives that need to be defined with respect to each activity performed under Identity Management:

• Identification – the security control process that creates an entity and verifies the credentials of the individual, which together form a unique identity for authentication and authorization purposes
• Authentication – a security control process that verifies credentials to support an interaction, transaction, message, or transmission
• Authorization – a security control process that grants permissions by verifying the authenticity of an individual’s identity and permissions to access specific categories of information or functions exposed by a resource.
• Accountability – a security control process that records the linkage between an action and the identity of the individual or role who has invoked the action, thus providing an evidence trail for audit or non-repudiation purposes
• Audit – a security control process that examines data records, actions taken, changes made, and identities/roles invoking actions which together provide a reconstruction of events for evidence purposes

All the control objectives above serve the requirement to provide an auditable chain of evidence.

Identity management control processes should have an input, one or more control activities, an output, feedback, management monitoring, and an overall audit appraisal activity to ensure that they are fit-for-purpose. The starting point is an individual who is enrolled into an organization and subsequently acts in a function or role in the organization. The individual may be an employee, partner, or contractor, or third party. The output is the appropriate degree of policy enforcement and individual accountability for the business activity. Within the controls, the threats and vulnerabilities constituting the business risk must be addressed and assessed. These include business, legal, and technical aspects.

Like with any systems, the following are the key non-functional requirements an Identity Management infrastructure should aim to offer.

• Being more responsive and secure
• Interoperability with a multitude of systems requiring identity information.
• Support for multiple authentication mechanisms, like two factor, bio-metric, etc.
• Interfaces and APIs for automation which could result in reduction in operational costs.

A governance framework would not be complete if it does not define the measurements that indicate the efficiency and effectiveness. The following are some of the metrics that could be considered:

• Password Reset volume – A well managed Identity Management System is expected to considerably reduce the help desk calls on forgotten passwords. As such a measure of this activity could be a key metric to establish that there is a considerable saving in such help desk activities.
• Number of distinct credentials per user – With Single Sign On implemented, there should be only one distinct credential per user.
• Average time taken for each of the identity management functions could be another useful metric to establish that the investment is worth.

References:

White paper on Identity and Access Management published byThe Open Group

More related reading:

Liberty Alliance Project