BNY Mellon Software Glitch: Time to make SysTrust mandatory?

As was widely reported in the business press, BNY Mellon experienced a technical glitch that affected its ability to price mutual funds accurately. Based on the press release from one of the affected funds, the problems started on Monday August 24th, where one of BNY Mellon's system "InvestOne" managed by SunGard was pricing about 800 mutual funds inaccurately.

So what was the cause of this fiasco?

According to CNN, "BNY Mellon outage occurred after a SunGard accounting system it uses became "corrupted" following an upgrade. A back-up also failed."

Normally, this type of thing will force the party experiencing the breach intense scrutiny over what went wrong. However, as I went through the timeline posted by the company, I found (reading between the lines) that they did a number of things right, such as:

• Incident Management Communication Plan: One of the aspects of incident management is communicating to the public and making them aware of the issue. As it can be seen, the company posted details on estimates as to when the system would be ready as well as what was the source of the delay when they didn't meet their estimated deadline. 
• Contingency planning: BNY Mellon was able to get the Net Asset Values (NAVs) "using a secondary and more manual process" to the clients without the production system. According to WSJ, this involved "mobilizing more than 100 accountants to manually calculate the values of thousands of fund securities". The secondary procedure was not perfect as small differences were noted between the fund and, as reported the WSJ, this could expose the bank to "potential liability from the resolution of trades that may have been made during the pricing fog".  
• Effective Service Level Agreement (SLA) with SunGard: Based on the close coordination with SunGard, it appears that they were able to get SunGard to eventually rebuild the system "from scratch". 

That being said, there is always room for improvement. When I was reflecting on this, I speculated that this was another case of inadequate testing of the system upgrade. However, according to SunGard, this was not the case. As they noted on their website:

"The issue appears to have been caused by an unforeseen complication resulting from an operating system change performed by SunGard on Saturday, August 22nd. This maintenance was successfully performed in a test environment, per our standard operating procedure, and then replicated in SunGard’s U.S. production environment for BNY Mellon. This change had also been previously implemented, without any issues, in other InvestOne environments. Unfortunately, in the process of applying this change to the SunGard production environment of InvestOne supporting BNY Mellon’s U.S. fund accounting clients, that environment became corrupted. Additionally, the back-up environment hosted by SunGard, supporting BNY Mellon’s U.S. fund accounting clients, was concurrently corrupted, thus impeding automatic failover. Because of the unusual nature of the event, we are confident this was an isolated incident due to the physical/logical system environment and not an application issue with InvestOne itself."

Given my background as a CA, CPA and CISA, I have always thought it is an odd contradiction that we expect infrastructure (road, dams, bridges, etc.) to be certified by engineers to be in working order (key word is expect, as John Oliver notes in the video below, this is not exactly up to snuff!), but do not have the same expectations for the technology that runs the Information Age.

And that's where I have always proposed that it is necessary to have a framework like SysTrust (now SOC2 and SOC3) in place that requires companies to ensure that their systems are reliable: secure, available, and able to process information without messing it up.

Based on the experience between SunGard and BNY Mellon, I think it actually proves the case. Although companies, like SunGard, likely have such controls in place it is beneficial to others to have a second set of eyes on those controls, ensuring that they are in place, are designed effectively and are operating effectively. The reason is that with such mandatory audits in place, it will allow for the circulation of best practices through such audits. This occurs in the financial auditing world through "management letter points".

One other area that we should explore is the total impact of this error, as it will give insights into the "total impact of failed IT controls". This will be the topic of the next blogpost.

Auditing the Algorithm: Is it time for AlgoTrust?

This is the third instalment of a multi-part exploration of the audit, assurance, compliance and related concepts brought up in the book,  Big Data: A Revolution That Will Transform How We Live, Work, and Think (the book is also available as an audiobook and hey while I am at it, here's the link to the e-book ).  In the last two posts we explored the more tactical examples of how big data can assist auditors in executing audits resulting in a more efficient and effective audit. The book, however, also examines the societal implications of big data. In this instalment, we look explore the role of the algorithmist.

Why do we need to audit the "secret sauce"?
When it comes to big data analytics, the decisions and conclusions the analyst will make hinges greatly on the underlying actual algorithm.  Consequently, as big data analytics become more and more part of the drivers of actions in companies and societal institutions (e.g. schools, government, non-profit organizations, etc.), the more dependent society becomes on the "secret sauce" that powers these analytics. The term "secret sauce" is quite apt because it highlights the underlying technical opaqueness that is commonplace with such things: the common person likely will not be able to understand how the big data analytic arrived at a specific conclusion. We discussed this in our previous post as the challenge of explainability, but the nuance here is that is how do you explain algorithms to external parties, such as customers, suppliers, and others.

To be sure this is not the only book  that points to the importance of the role of algorithms in society. Another example is "Automate This: How Algorithms Came to Rule Our World" by Chris Steiner, which (as you can see by the title) explains how algorithms are currently dominating our society. The book bring ups common examples the "flash crash" and the role that "algos" are playing on Wall Street in the banking sector as well as how NASA used these alogrithms to assess personality types for its flight missions. It also goes into the arts. For example, it discusses how there's an algorithm that can predict the next hit song and hit screenplay as well as how algorithms can generate classical music that impresses aficionados - until they find out it is an algorithm that generated it! The author, Chris Steiner, discusses this trend in the follow TedX talk:



So what Mayer-Schönberger and Cukier suggest is the need for a new profession which they term as "algorithmists". According to them:

"These new professionals would be experts in the areas of computer science, mathematics, and statistics; they would act as reviewers of big-data analyses and predictions. Algorithmists would take a vow of impartiality and confidentiality, much as accountants and certain other professionals do now. They would evaluate the selection of data sources, the choice of analytical and predictive tools, including algorithms and models, and the interpretation of results. In the event of a dispute, they would have access to the algorithms, statistical approaches, and datasets that produced a given decision."

The also extrapolate this thinking to an "external algorithmist": who would "act as impartial auditors to review the accuracy or validity of big-data predictions whenever the government required it, such as under court order or regulation. They also can take on big-data companies as clients, performing audits for firms that wanted expert support. And they may certify the soundness of big-data applications like anti-fraud techniques or stock-trading systems. Finally, external algorithmists are prepared to consult with government agencies on how best to use big data in the public sector.

As in medicine, law, and other occupations, we envision that this new profession regulates itself with a code of conduct. The algorithmists’ impartiality, confidentiality, competence, and professionalism is enforced by tough liability rules; if they failed to adhere to these standards, they’d be open to lawsuits. They can also be called on to serve as expert witnesses in trials, or to act as “court masters”, which are experts appointed by judges to assist them in technical matters on particularly complex cases.

Moreover, people who believe they’ve been harmed by big-data predictions—a patient rejected for surgery, an inmate denied parole, a loan applicant denied a mortgage—can look to algorithmists much as they already look to lawyers for help in understanding and appealing those decisions."

They also envision such professionals would work also work internally within companies, much the way internal auditors do today.

WebTrust for Certification Authorities: A model for AlgoTrust?
The authors bring up a good point: how would you go about auditing an algo? Although auditors lack the technical skills of algoritmists, it doesn't prevent them from auditing algorithms. The WebTrust for Certification Authorities (WebTrust for CAs) could be a model where assurance practitioners develop a standard in conjunction with algorithmists and enable audits to be performed against the standard. Why is WebTrust for CAs a model? WebTrust for CAs is a technical standard where an audit firm would "assess the adequacy and effectiveness of the controls employed by Certification Authorities (CAs)". That is, although the cryptographic key generation process is something that goes beyond the technical discipline of a regular CPA, it did not prevent the assurance firms from issuing an opinion.

So is it time for CPA Canada and the AICPA to put together a draft of "AlgoTrust"?

Maybe.

Although the commercial viability for such a service would be hard to predict, it would help at least start the discussion around of how society can achieve the outcomes Mayer-Schönberger and Cukier describe above. Furthermore, some of the ground work for such a service is already established. Fundamentally, an algorithm takes data inputs, processes it and then delivers a certain output or decision. Therefore, one aspect of such a service is to understand whether the algo has "processing integrity" (i.e. as the authors put it, to attest to the "accuracy or validity of big-data predictions"), which is something the profession established a while back through its SysTrust offering. To be sure this framework would have to be adapted. For example, algos are used to make decisions so there needs to be some thinking around how we would identify materiality in terms of  total number of "wrong" decisions as well as defining "wrong" in an objective and is auditable manner.

AlgoTrust, as a concept, illustrates not only a new area where auditors can move its assurance skill set into an emerging area but also how the profession can add thought leadership around the issue of dealing with opaqueness of algorithms - just as it did with financial statements nearly a century ago.

Can we live in the cloud? Prof Jeff Jarvis intends to find out

On This Week in Google (TWIG) episode 169, Jeff Jarvis, professor of journalism at CUNY, announced that he will be attempting to live only in the cloud and abandoning the comforts of offline desktops.  He recently moved to the Android eco-system (i.e. for his mobile device and tablet), which he accredits to Google's wide range of services from maps to Google Docs. Taking it to "whole nother level", Jeff is planning to live only in the cloud once he gets his hands on Samsung's ultra-cheap Chromebook, which is expected to retail for $249. The Chromebook (as its names suggests) is based on Google's Chrome OS, where the OS is basically the Chrome browser. Here's the ad in case you missed it:


As illustrated in the ad, the concept is that the Chromebook is something that everyone and anyone can use. The premise is: if you primarily do everything in the browser, then you really don't need a full laptop. A few years ago, as Leo Laporte pointed out in the episode, this experiment by the way of netbooks failed. Does Jeff have a fighting chance or will Leo tell Jeff "I-told-you-so" after Jeff experiment ends? Well, I think Jeff does have a fighting chance. Firstly, cloud computing has matured significantly since netbooks have hit the scene. Secondly, people are now accustomed to using tablets and smartphones as a way to get things done.

In a way the Chromebook represents an intersection between the trend of cloud computing and thin client devices and taking technology back to the early years of computing, where users had to "dial-in" from their "dumb terminals" into powerful mainframes. Except the Chromebook,smartphones, and tablets are replacing the dumb terminals, while the cloud computing service providers are replacing the mainframe.

Why should information security & privacy professionals care about this?

It is really about the price point. If Jeff Jarvis can successfully move to the cloud with this device, it means that the economics of the consumerization of IT has arrived. Think of a 10-person small business that is starting up. It really just needs email and office productivity apps for their clients. The IT cost would be $2500 for the hardware and then recurring cost of $500 a year for the Google Apps. The traditional  Dell laptop + MS Office license would cost about $6480 upfront + the cost of an email server + the IT resources an effort to maintain/patch the laptops and the server.

In terms of data redundancy, one could argue that all the data is on the cloud so it's actually safer. Theoretically, if the owner loses their Chromebook, they can just change their password and then the Chromebook is essentially just a "dumb" piece of hardware with no data. And as illustrated by these stats, this is no small benefit. Of course, cloud computing does have its risks as mentioned on a previous blog post and this publication (which I co-authored for the CICA). It's not that the risks in the cloud are insurmountable, but they are different then the ones we are accustomed to dealing with.

From a usability and information risk perspective I would ask these questions to Jeff Jarvis about his experiment:

• Printing: What are the hiccups in terms of producing and printing formatted documents? What I am thinking about are the mundane things like resumes, reports and the like. 
• Working with Luddites: How do you work with others that are not in the cloud? Sometimes working with a colleague the most efficient way to transfer a number of documents is via USB, especially when the other party does not have Internet access (e.g. think of locked down company laptops). 
• Handling Sensitive Data: What is the sensitivity of the data that is being on the cloud? For example, we keep private things like tax files that contain SSNs, SINs, income, etc offline. So how would one keep such things private or is it matter of just living in public? For readers that are unfamiliar with Jeff Jarvis, he takes "what's the harm approach and has written two books (click here and here) on the topic of being more open and social with one's information. But I hope he can appreciate not everyone uses his "privacy settings" :)
• Trusting cloud providers: What due diligence does someone do before trusting a cloud provider? I suppose this is a "leading question".  Accounting associations in Canada (i.e. the CICA) and the US (AICPA) have established Service Organization Control (SOC) Reports. These reports replaced the SAS 70 Type II reports in the US and Section 5970 Reports in Canada. So do you need this type of assurance before dealing with companies? Going back to the tax return example, one solution would be to use cloud-based tax services. But how do you establish trust that this information is appropriately. One may attribute my repetitive use of the tax return info to the fact that I am an accountant. However, to be fair Gina Trapani on a previous episode of TWIG did point out an accountant should not be putting tax info on the cloud unless it was encrypted. 
• Securing data on the lost Chromebook. If the Chromebook is lost, what are the precautionary measures the person has to take? In other words, the theory meet reality. 
• Making local backups:  Currently, we back from offline to the cloud, but how does this work in reverse? The reason this is important is illustrated by Mat Honan's Apple iCloud account getting hacked and watching helplessly as his data got deleted. 
• Working without internet access: How many times does the lack of internet access due to being in a subway or non-WiFi become an obstacle to being productive?
• Working through cloud outages: What happens if there is a disruption at the cloud provider or underlying infrastructure? Jeff lives in NY (and judging by his tweets; he's doing okay), so he does have some experience dealing with such a scenario given the disaster brought to his area by Hurricane Sandy. 

Assuming Jeff actually does gets his Samsung Chromebook and goes through with this experiment, I will post an update to this post.

MS Office goes Cloud: Quick overview of benefits and things to watch out for

Earlier this month, CNET's Mary Jo Foley reported on Microsoft's move to Office 2013. As noted on a previous blog post, this is a huge year for Microsoft as it moves to the tablet-centric  Windows 8 operating system. Well, they seem to be doubling down on dramatic shifts as they launch a SaaS offering of their infamous Office productivity suite; Office 365. Mary Jo reports that Microsoft will be giving a choice between purchasing Office 2013 as "normal" or as a subscription to its cloud version of the software. To sweeten the offer Microsoft is offering the following extras (credits: Mary Jo and Paul Thurrott): 

• Ability to log-in to 5 different PCs or Macs 
• Access to Word, Excel, Powerpoint, OneNote, as well Access, Publisher and Outlook
• 60 Skype World Minutes a month
• 20 GB of SkyDrive storage
• Update on security and other patches
• Access to new functions through the subscription period (i.e. you don't need to wait for the next version)

In contrast, the standard PC-installed version of Office 2013 can only be installed on one machine. Also, to get access to Access, Publisher and Outlook you need to Professional version (Mary Jo has a great table here that explains the different options). 

Office 365 Home Premium is $99.99/year, which covers an "entire household" (i.e. Paul Thurrott explains that it is not tied to a single individual, but can be used any person located at that address). Assuming that this will be same price in Canada, this would amount to $9.42/month (including HST) which is cheaper than two venti lattes at Starbucks. This is in contrast to Office 2013 Professional, which retails for 399.99+HST (and 139.99+HST for the Home & Student version, which includes Word, Excel, Powerpoint, and OneNote). 

However, the big story here is that Microsoft getting the average user  - to the Cloud! (Oh, yes – it was Microsoft that came up with those terrific ads didn't they?). Some may say that this is yesterday's news because Google Docs  has already brought cloud-based office productivity. Although that may be true, if you ask my students they're using Google Docs to collaborate but still rely on MS Office to print a report or assignment. And of course when they go on their work terms, the firms are still using MS Office (so they need to know how it works and be able to use it well).   

In other words: Is the world ready for moving their recipes, financial budgets, and other personal documents to the cloud? 

For those that want the full low down on cloud, they can download this whitepaper from the CICA, which I wrote with Yvon Audette of KPMG. Alternatively, here is a short list of things that you can talk to your friends or whoever that are wondering what happens if they decide to go to go with Office 365 or another cloud based app.

Pay for what you use: In terms of benefits, MS has really sweetened the pie with the extras they noted above. The other implicit benefit is that you are not paying for a static piece hardware upfront. Furthermore, if you decide to change your mind later on you will be out only $100 instead of $400. For example, to buy Office  Professional you have to fork over $400 on the spot, where as with Office 365 you pay as you go (i.e. $100 per year). So if you decide a year from now that you don't use all the extras that Office 365 comes up (i.e. let's say you are not using the extra software, such as Publisher, Access, Skype, etc) you can buy the Starter version or switch to an open source alternative. 

The Cloud Can Go Down, but so can your laptop: There have been cases of cloud outages, as I noted in my last post. Consequently, you should create a local backup of your files from Office 365, so that they are accessible off of the cloud (I am hoping Microsoft will make this easy) and won't get corrupted if there is a problem at Microsoft. However, let's be honest - what's more likely to go down Microsoft or your own laptop? The advantage of Office 365 is that if your laptop goes down, you can always access it from another laptop. In other words, your data is no longer tied to your machine.

You have less control, but you've handed it over to Microsoft (who should know a little bit about good computing practices): It should be clear that you are handing over your files to Microsoft to manage for you. But this may be a good thing, as they may do a better job than you. For example, if you don't do local backups (as you should), then Microsoft likely does. According to this link, they perform an ISO 27001 audit (click here to see what that covers) as well as HIPAA, FISMA, and EU Model Clauses. The certification that is absent is the new SOC 2 (see here for the difference between SOC 2 and SOC1. SOC 1 replaced the SAS 70 Type II reports, which outsourcers previously used and abused).

Terms of service (ToS), assume nothing: In general, cloud service providers have an army of lawyers to indemnify them from pretty much everything. So you should assume if anything goes wrong it's tough luck for you. Also, beware on what they say in terms of who owns the data (ZDNet did an analysis last year for online storage, we hope they update it for the new Office 365). According to this post, Microsoft pays back money for downtime for the Office 365 they were offering to businesses - but it is unclear whether they would do the same for consumers. 

Is a hacker also using Office 365? Amazon's cloud service, EC2, was used by hackers to launch the infamous attack on Sony's PSN. Security researchers were also able to spy on fellow "tenants". So what do these two facts add up to? Hackers will try to see what  vulnerabilities exist in Office 365 to exploit to get data from other users. That being said, hackers are mostly after credit card data and it may be more trouble than it's worth to mine terabytes of cake recipes and essays on Shakespeare to find what they are looking for (but 'big data tools' do make this easier). 

Privacy: accidental disclosures and the reality of law enforcement. In addition to nefarious individuals lurking on the internet, there is a risk that something will go wrong and the wrong user will get access to your documents. For example, Microsoft's precursor to Office 365 (known excitingly as BPOS) experienced precisely this kind of breach (to be fair here is MS's defense). In terms of privacy, the way the privacy rules works is that if the provider tells you in the ToS that they will hand over things to law enforcement then they are covered from a privacy compliance perspective. (See the Privacy Commissioner's handling of the complaints against CIBC). Furthermore, as noted in this article both American and Canadian law enforcement and other agencies can access what you put on Office 365 and they don't need to do tell you about it. 

With Microsoft's push to the cloud, it will be interesting to "consumer outsourcing" works out. For example, how will the masses react to an outage? Will grade school teachers accept the excuse that the "cloud ate my homework"? Or will we be surprised at how adept people are to the new realities of the cloud? For example, people nowadays have camera free parties to manage the risk of the 24-7 surveillance world we live in due to social networks. Practically, consumers can use free open source alternatives to keep their personal documents offline and use Office 365 for things that they don't consider sensitive or to meet the demands of employers/customers and some of these providers are keenly working to make their offerings interact with Office 365. However, the problem is that if they are used to using Excel offline to keep their budgets are they really going to switch to the open source alternative? I guess we will wait and see what happens.