Cafe X and Amazon Go: Auditing a robot-operated store?

By now you've probably heard of the robot-barista - Cafe X.  If not check out this video from Wired, where David Pierce walks us not only through how the robot will make your latte, but why he thinks it better than the human alternative:



Amazing isn't it?

In a presentation I did last year on how these forces of automation could impact auditing & accounting, I noted it's easier to see how technology disrupts someone other than you.

And so it looks like baristas have met their match.

As Pierce notes in the video, the inconvenience of dealing with imperfect people is something that most people want to avoid in the rat-race we live in: who wants the barista to remake your coffee 11 times as he says? ;) 

The Wired article also notes that Cafe X is 'high-quality at a cheaper price': 

"Surprisingly delicious coffee, starting at $2.25—cheaper than you’d find at Sightglass or even Starbucks. Cafe X’s location in the corner of the Metreon may not entice you out of your daily routine."

Amazon Go: Walkthrough Technology 
Amazon has also wowed the "techthusiasts" out there with their cashier-less store concept:



In the FAQ section, Amazon summarizes how this cashier-less store works:

"Our checkout-free shopping experience is made possible by the same types of technologies used in self-driving cars: computer vision, sensor fusion, and deep learning. Our Just Walk Out Technology automatically detects when products are taken from or returned to the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. Shortly after, we’ll charge your Amazon account and send you a receipt."

Although this has the potential to revolutionize retail, Amazon has experienced some setbacks of late. The store can allegedly only handle 20 people at a time. So there maybe some kinks to work out before this goes mainstream.

Obviously, this could have a massive impact on entry level jobs: most of us who were young a while ago relied on these McJobs for spending money and funding our college/university tuition. They also gave students some practical work experience to help land a career accounting profession ;)

But let's save this discussion for a future post.

How would you audit cashier-less stores, like Cafe X or Amazon Go?

The retail industry has been a manual intensive industry that requires cashiers, stock room personnel and the like. Such a process naturally requires policies and procedures (aka internal controls) that ensure that merchandise makes it from the shelf to the cash register and into the customers possession. And there are those anti-theft mechanisms to prevent shoplifting as well. In the industry, "shrinkage", the amount of merchandise that is stolen, robbed, damaged, etc, is estimated by the National Retail Federation to be 1.38% of sales or $45.2 billion for 2015.

Cafe X and Amazon Go offer a glimpse into how automating traditional businesses can alter these fundamental risks that impact the way we go about conducting our financial audits.

With Cafe X, shrinkage is almost eliminated as there is no humans involved in the production process. Once the kiosk is loaded up with cups, coffee, syrup, sugar, milk, etc. the system is essentially fully automated - no manual intervention by baristas or customers.

Amazon Go, on the other hand, uses a whole lot of automation that is watching and analyze every move of the customers (and employees) throughout the store. Consequently, this would not be the store to steal from! And let's not forget Amazon is experimenting with those drones and are we really sure that they are unarmed?


Given this level of automation of the actual business process and controls, could auditors stick to the tried, tested and true retail audit procedures? Or would this enable a more automated approach?

I was directly involved with the recent test-audit of the blockchain involving loyalty points. One of the realities of auditing such exponential technologies is that it makes controls testing a must. For example, for the financial auditor to rely on the digital signatures there needs to be some testing around the wallets to ensure that the signatures are reliable.

Consequently, testing such automated stores would require either a SOC2 or modified SOC report to meet the needs of such a store. For example, the SOC2 would need to have some way of having comfort of how the stock and inventory gets loaded into the store. Likely the auditor would rely on the automated process which the store uses to replenish stock, but it's that hand off between the delivery person (assuming it's still human) that would be the area there is a risk of shrinkage. For example, how does legitimately damaged inventory get accounted for at that point? Whatever process and controls Amazon/Cafe X put in place would need to be tested from a controls perspective.

For the substantive component, I think that's where things get interesting: enter the "embedded audit module". This concept has been around since at least 1989. The idea is that the auditor installs independent software onto the client's system and then transmits it back to the auditor, who uses it as a basis for conducting the necessary audit procedures and tests. The core idea is that the auditor has full control over such a system and the client cannot tamper with the code.

What would be relatively straightforward would be the data capture-component: sales data, stock data, spoilage, etc. would be uploaded from the automated store right into the auditor's system. But this then requires the additional step of verifying the data to independent source documents (e.g. invoices, purchase orders, etc.). In other words, the audit procedure would still require manual intervention as the auditee would need to send this information back to the auditor to complete their audit.

Where I think the audit innovation would be is exploring how video footage can act as a substitute for physical/direct observation by the auditor. That is, could the auditor install a video camera in the automated store as a part of the EAM that would then act as actual independent audit evidence of the actual sale or purchase? For example, in the Cafe X example the auditor could actually use the footage and the visual software to count the cups sold that day and reconcile that to the sales data transmitted back from the EAM for the day?

Although one can argue such transactions are not material and therefore such procedures are overkill.

However, I think now is the right time to conduct experiments and test audits to see whether we can reinvent the classic audit to meet the technology of today. In a future post, we will explore what this means broadly for jobs and more specifically how this could impact the profession.

Author: Malik Datardina, CPA, CA, CISA. Malik works at Auvenir as a GRC Strategist that is working to transform the engagement experience for accounting firms and their clients. The opinions expressed here do not necessarily represent UWCISA, UW, Auvenir (or its affiliates), CPA Canada or anyone else

Virtual Personal Assistants: How far will they go? Part 2

In the last post, I spoke about the advent of the Virtual Personal Assistants (VPAs) in terms of Gartners predictions as to where they will go and how popular culture sees them coming to enable our lives.

For the second part of this post, I wanted to talk about my first work experience - ever - with a fully virtual assistant.

Let me set the context.

In the course of my work, I was dealing with a vendor who was trying to arrange a meeting with us through his personal assistant, Amy Ingram.

So we were going back and forth to fix a date and time for the conference call.

I responded to the initial request as follows:

"Hi Amy,

Actually out of town on Tuesday; Thursday is open though. Does that work with you?"

"Her" response was (using Billy as a pseudonym):

"Hi Malik,

I'm sorry, but that time doesn't work for "Billy".

How about Wednesday, Jun 22 at 11:30 AM EDT? "Billy" is also available Wednesday, Jun 22 at 3:00 PM EDT or Thursday, Jun 23 at 9:00 AM.

Amy"


When I read Amy's response, I thought to myself something like: "I told her that Thursday is open, so why did she say that doesn't work for the "Billy"?"  But I thought something like "whatever" and just responded with:

"Thursday at 9 am works, thanks"

To which Amy responded:

"Hi Malik,

Thanks for letting me know.

I'll send out an invite once I've confirmed a time with "Jim".

Amy"

[Jim is my colleague; true name hidden for confidentiality purposes]

Eventually, it dawned on me: I wasn't dealing with a person, but a robot!

And then it hit me: the future is here.

The one thing that I realized through my interaction is how forgiving I was about the error because I thought the thing on the other side was human: everyone makes mistakes and so it's no big deal that "she" didn't get that I was open on Thursday.

This has a deeper implication on how "knowledge work" gets automated.

When we gauge machines for the ability to perform cognitive tasks, such as booking meetings, we should be careful as to how good is good enough for us to work with machines instead of humans. As we can see based on my interaction, they don't need to be perfect - they just need to get the job done.

In my interaction above, we were able to schedule a meeting and the fact "she" didn't understand that I had told her Thursday was open had no real consequence on the overall role "she" was playing. The meeting eventually got booked and that was that.

Ironically, I realized that I had already come across Amy at the DLD Conference in NY that had attend a few weeks earlier.

Dennis Mortensen (Founder of x.ai.), describes the challenge of setting up meeting and how this technology can solve the problem (profanity alert!):

His talk starts 5m47s:

As Dennis mentions, it's a very basic problem but at the same time it's so complicated. Specifically, the challenge with dealing with politeness: it's hard for AI to parse through this and understand the substantive facts that pertain to setting up the meeting. If we take a look at my response, we can see the challenges first hand:

• When I said I was out of town that the AI had to understand that meant I am not available. 
• I did not include Wednesday as a date that was possible so that implies that I'm also not available that day.
• When I stated I was open on Thursday, I meant I was available all day. 

So what does this mean for jobs? Are accountants going to be replaced by Amy one day?

It's actually shows the level of complexity involved in the most basic of human interactions and how much more complex it would be to train AI in terms of doing even the most basic of auditing procedures - at least for now. 

Dennis actually made a good point about this in the Q&A portion of the discussion as it relates to jobs. The other presenter noted how he sees massive displacement as a result of AI; specifically in the truck driving industry. Dennis, on the other hand, was a bit more optimistic. He noted that what tools like his will do is essentially give assistants to people who don't have assistants. For example, the vendor we were dealing likely wouldn't have hired an assistant to help book appointments. 

And I think that's where auditors and accountants need to actually see how AI assistants, like Amy Ingram, can help with automating those mundane tasks that none of likes to do.  

Low Decision Agility: BigData's Insurmountable Challenge?

Working in the field of data analytics for over decade there is one recurring theme that never seems to go away: the overall struggle organizations have with getting their data in order.

Courtesy of this link. 

Although this is normally framed in terms of data quality and data management, it's important to link this back to the ultimate raison d'etre for data and information: organizational decision making. Ultimately, an organization has significant data and information management challenges it culminates into a lack of "decision agility" for executive or operational management. I define decision agility as follows:

"Decision agility is the ability of an entity to provide relevant information 

to a decision maker in a timely manner." 

Prior to getting into the field, you would think that with all the hype of the Information Age it would be easy as pressing a button for a company to get you the data that you need to perform the analysis you need to do. However, after getting into the field, you soon realize how wrong this thinking: most organizations have low-decision agility.

I would think it is fair to say that this problem hits those involved in external (financial) audits the hardest. As we have tight budgets, low-decision agility at the clients we audit makes it cost-prohibitive to perform what is now known as audit analytics (previously known as CAATs). Our work is often reigned in by the (non-IT) auditors running the audit engagement because it is "cheaper" do the same test manually rather than parse our way through the client's data challenges

So what does this have to do with Big Data Analytics?

As I noted in my last post, there is the issue of veracity - the final V in the 4 Vs definition of Big Data. However, veracity is part of the larger problem of low decision agility that you can find at organizations. Low-decision agility emerges from a number of factors and can have implications on a big data analytics initiative at an organization. These factors and implications include:

• Wrong data:  Fortune, in this article, notes there is the obvious issue of "obsolete, inaccurate, and missing information" data records itself. Consequently, the big data analytics initiative needs to assess the veracity of the underlying data to understand how much work needs to be done to clean up the data before meaningful insights can be drawn from the data. 
• Disconnect between business and IT: The business has one view of the data and the IT folks see the data in a different way. So when you try to run a "simple" test it takes a significant amount of time to reconcile business's view of the data model to IT's view of the data model. To account for this problem there needs to be some effort in determining how to sync the user's view of the data and IT's view of the data on an ongoing basis to enable the big data analytic to rely on the data that sync's up with the ultimate decision maker's view of the world.  
• Spreadsheet mania: Let's fact it: organizations treat IT as an expense not as an investment. Consequently, organizations will rely on spreadsheets to do some of the heavy lifting for the information processing because it is the path of least resistance. The overuse of spreadsheets can be a sign of an IT system that fails to meets the needs of the users. However, regardless of why they are used, the underlying problem is dealing with these vast array of business-managed applications that are often fraught with errors and outside the controls of production system. The control and related data issues become obvious during compliance efforts, such as SOX 404 or major transitions to new financial/data standards, such as the move to IFRS. When developing big data analytics, how do you account for the information trapped in these myriad little apps outside of IT's purview? 
• Silo thinking: I remember the frustration of dealing with companies that lacked a centralized function that had a holistic view of the data. Each department would know it's portion of the processing rules, etc. but would have no idea of what happened upstream or downstream. Consequently, an organization needs to create a data governance structure that understands the big picture and can identify and address the potential gaps in the data set before it is fed into the Hadoop cluster.  
• Heterogenous systems: Organizations with a patch-work of systems require extra effort from getting the data formatted and synchronized. InfoSec specialists deal with this issue of normalization when it come to security log analysis: the security logs that are extracted from different systems need to have the event IDs, codes, etc. "translated" into a common language to enable a proper analysis of events that are occurring across the enterprise. The point is that big data analytics must also perform a similar "translation" to enable analysis of data pulled from different systems. Josh Sullivan of Booz Allen states: "...training your models can take weeks and weeks" to recognize what content fed into the system are actually the same value. For example, it will take a while for the system to learn that female and woman are the same thing when looking at gender data. 
• Legacy systems:  Organizations may have legacy systems which do not retain data, are hard to extract from and difficult to import into other tools. Consequently, this can cost time and money to get the data into a usable format that will also need to be factored into the big data analytics initiative.
• Business rules and semantics: Beyond the heterogenity differences between systems there can also be a challenge in how something is commonly defined. A simple example is currency: an ERP that expand multiple countries the amount reported may be in the local currency or the dollar, but requires the metadata to give that meaning. Another issue can be that different user group define something different. For example, for a sale for the sales/marketing folks may not mean the same thing as a sale for the finance/accounting group (e.g. the sales & marketing people may not account for doubtful accounts or incentives that need to be factored in for accounting purposes). 

Of course these are not an exhaustive list of issues, but it gives an idea of how the reality of analytics is obscured the tough reality of state of data.  

In terms of the current state of data quality, a recent blog post by Michele Goetz of Forrester noted that 70% of the executive level business professionals they interviewed spent more than 40% of their time vetting and validating data. (Forrester notes the following caveat about the data: "The number is too low to be quantitative, but it does give directional insight.")

Until organizations get to a state of high decision agility - where business users spend virtually no time vetting/validating the data - organizations may not be able to reap the full benefits of a big data analytics initiative. 

Worth mentioning: KPMG's take on the state of tech in the audit profession

In a recent post (as in just this week) on Forbes, KPMG's  James P. Liddy who is the Vice Chair, Audit and Regional Head of Audit, Americas put out a great post that summarizes the current state of analytics in financial audits.

He diplomatically summarizes the current state of the financial audit as "unchanged for more than 80 years since the advent of the classic audit" while stating "[a]dvances in technology and the massive proliferation of available information have created a new landscape for financial reporting. With investors now having access to a seemingly unlimited breadth and depth of information, the need has never been greater for the audit process to evolve by providing deeper and more relevant insights about an organization’s financial condition and performance –while maintaining and continually improving audit quality." [Emphasis added]

For those that have started off our careers in the world of financial audit as professional accountants and then moved to the world of audit analytics or IT risk management, we have always felt that technology could help us to get audits done more efficiently and effectively.

I was actually surprised that he stated that auditors "perform procedures over a relatively small sample of transactions – as few as 30 or 40 – and extrapolate conclusions across a much broader set of data". We usually don't see this kind of openness when it comes to discussing the inner-workings of the profession. However, I think that discussing such fundamentals is inevitable given those outside the profession are embracing big data analytics in "non-financial audits". For example, see this post where I discuss the New York City fire department's use of big data analytics to identify a better audit population when it comes to identifying illegal conversions that are a high risk and need to be evacuated.

For those that take comfort in the regulated nature of the profession as protection of disruption, we should take note of how the regulators are embracing big data analytics. Firstly, the SEC is using RoboCop to better target financial irregularities. Secondly, according to the Wall Street Journal, FINRA is eyeing an automated audit approach to monitoring to risk. The program is known as "Comprehensive Automated Risk Data System" (CARDS). As per FINRA:

"CARDS program will increase FINRA's ability to protect the investing public by utilizing automated analytics on brokerage data to identify problematic sales practice activity. FINRA plans to analyze CARDS data before examining firms on site, thereby identifying risks earlier and shifting work away from the on-site exam process". In the same post, Susan Axelrod, FINRA's Executive Vice President of Regulatory Operations, is quoted as saying "The information collected through CARDS will allow FINRA to run analytics that identify potential "red flags" of sales practice misconduct and help us identify potential business conduct problems with firms, branches and registered representatives".

As a result, I agree with Mr. Libby: sticking to the status quo is no longer a viable strategy for the profession.