Friday, May 14, 2010

Product Review: Mark McClish's Statement Analyzer

This in-depth product review covers Statement Analyzer software created by Mark McClish. Mark McClish is a former U.S. Marshal with more than 25 years of experience in law enforcement. He is one of the most influential name in statement analysis. I've had the pleasure to review his software and I have tested it on over 200 texts, including the articles and comments on my blog, apologies, emails, incident reports and various other open statements. The main components of this software are reviewed with an outline of their high and low points.




An Overview of Statement Analyzer
Statement Analyzer promises to analyze any statement and assist you in determining whether a subject is truthful or deceptive for a mere 35$US. I have received the CD within a reasonable time frame. The international shipping is relatively standard with approximately two (2) weeks.

Statement Analyzer comes in a shrink wrapped CD case with a nice black and white instructions booklet. The booklet and installer instructions are somehow unusual in the world of technologies but nothing that cannot be understood. The installation went flawlessly to the exception it is not possible to choose an installation directory and the software is installed in C:\STMTANAL\.

This small application takes approximately 17 mb on disk. It comes with an annoying and useless copy protection involving its user to have the CD in the drive while using the software. I believe it is possible to order a download-only version.
DETECTING DECEPTION The software is on version 10.01 but it feels like a 1.01. I have mentioned about deceptive version numbers in my article Rich Internet Application. This rule does not always apply to a software that has evolved through multiple operating systems. For example, the first version of Wordperfect for Windows was 5.1 but it was not deceptive because it was a continuation, or a port, of Wordperfect for DOS. It may however apply to softwares rewritten from the scratch.

User Interface and Usability
The user interface is solid and simple but it has usability issues and lacks of modern features. It has a look-and-feel given by a programming team that has never used the software. The user interface does need polishing.

The workflow is unusual in many ways because the user interface has several noticeable inconsistencies and annoyances. The software is provided with a Multiple Document Interface (MDI) but it is not possible to normally use multiple windows because they are modal, which means a window needs to be minimized or closed before opening another window.

The software does ask whether or not a user wishes to retrieve a saved statement every time a new statement button is clicked. Some standard shortcuts will not work in certain situations or if the focus is not on the right pane. The entire list of annoyances is easy to fix and I am sure that further updates and versions will take care of them.

There is no Unicode support which is a problem to insert statements with characters from foreign languages but also special characters such as “½”. Copy and paste may require some subsequent editing. The software uses ISO-8859-1 encoding.

The user interface is not particularly responsive. There is a noticeable delay between a user input and its outcome, and this comes off as a sluggish application. A more thoughtful and polished user interface would increase productivity. The workflow in particular has been neglected, if only to compare multiple statements side-by-side!

Printing Statement Analysis
The printing feature is good but not great. It has a custom printing preview window and it does feel outdated. The pagination is not properly rendered. The feature will not print on Adobe PDF printer but it will print on Microsoft XPS Document Writer. The resulting printed pages contain most of the information available in the software. I wish it would underline keywords in the statement and have some more statistics.

Statement Analysis Effectiveness
Statement Analyzer comes with a list of 763 keywords and 100 descriptions. This is a fairly good list but some important keywords are missing. For example, there are additional text bridges to these available in the software keyword database. The specificity of the dictionary is a shortcoming because it is mostly effective on incident reports. The software effectiveness is greatly reduced by its inability to manage (edit, add and delete) keywords and descriptions.

The descriptions are excellent and up to the reputation of Mark McClish. They are informative and most of them included examples. I wished it would have included at least two examples, one example that would trigger a red flag and another that would not, in order to better assist users in determining truthfulness in any given statement. In general, the descriptions available in the software are useful even to people who are experienced in detecting deception using statement analysis, including myself.

The software text processing engine is overly basic and provides only a case insensitive word matching algorithm. The sentence structures, semantics and all other goodies available in the field of Natural Language Processing (NLP) are unfortunately not available in Statement Analyzer. It simply lists any matching keywords regardless of any other factors, which sometimes becomes a distraction.

It does not match misspelled words and may lead a user to overlook misspelled keywords because 1) they would not be matched and thus not highlighted, and 2) they, along with other misspelled non-keywords, can be sign of emotional arousal or cognitive load and thus require further investigation.

The software comes with a much needed disclaimer in which it is stated Statement Analyzer is an investigative tool and the final judgement as whether or not a subject is truthful or deceptive should be left to its users. This is a good investigative tool in respect to the disclaimer.

Comparative Study: Jeffrey MacDonald

The question remains on how Statement Analyzer performs against a manual analysis performed by an expert. Is it a pocket version of Mark McClish? A comparative study will help to determine the effectiveness of the software.

The open statement given by Jeffrey MacDonald has been analyzed by Mark McClish on his website. I ran Statement Analyzer on the statement and collected data from its result. Furthermore, I have quantified the keywords in the result that were actually deceptive (a.k.a. red flags) according to me.

The statement contains 1815 words according to OO.o Writer. Statement Analyzer discovered a total of 72 keywords in the statement. The keywords appeared an overall total of 506 times. For example, the word “I” has appeared 191 — while not every “I” is deceptive, many of them in a statement denotes high stress. I have flagged 378 out of the 506 words as being deceptive, which is approximately 75% when including every “I” and 37% excluding them. Thus, the software reports that approximately 28% of the text is deceptive whereas the red flags account for 21%.

Mark McClish has determined that 70 expressions were deceptive in the statement and he has concluded the statement was highly deceptive. The software comes close to this assessment and it is effective considering the keywords flagged as deceptive. The software does indeed help to reach the same verdict as Mark McClish.

Comparative Study: Jessie Arbogast
The Jessie Arbogast's Shark Attack statement contains 408 words according to OO.o Writer. Statement Analyzer discovered a total of 19 keywords in the statement. The keywords appeared an overall total of 38 times. I have flagged 17 out of the 38 words as being deceptive, which is approximately 45%. Thus, the software reports that approximately 10% of the text is deceptive whereas the red flags account for 5%.

The analysis performed by the software leads to a dubious verdict. It is not possible to make a conclusive verdict with the result but the result is somewhat helpful in determining the truthfulness. The analysis performed by Mark McClish is better.

Comparative Study: Tiger Woods Apology
The Tiger Woods Apology contains 1544 words according to OO.o Writer. Statement Analyzer discovered a total of 57 keywords in the statement. The keywords appeared an overall total of 279 times. I have flagged 165 out of the 279 words as being deceptive, which is approximately 60% when including every “I” and 23% excluding them. Thus, the software reports that approximately 18% of the text is deceptive whereas the red flags account for 11%.

The analysis performed by the software does not lead to make a verdict. It does serve as an investigative tool but its function is overly limited by the structural complexity of an apology and its nature. Read more on statement analysis on apologies: Anatomy of an Apology, Detecting Deception in an Apology, and see tags [1] [2] [3].

Technology Behind The Curtain
Statement Analyzer is developed in Visual FoxPro, a data and database oriented programming language. For the younger readers, it is a technology that has been in decline since early 2000s. This is by all means an outdated but also unusual choice.

Database oriented programming languages are known to be particularly ineffective in natural language processing, let alone text processing. It will be extremely challenging to update the current software with appealing features, such as, dealing with sentence structures and semantics.

Furthermore, the last version of Visual FoxPro has been released in 2004. This means it will hardly be able to provide a full integration to newest operating systems, office suites, mail agents, web browsers, online chat applications and so on.

Public Relation and Customer Support
Mark McClish understands that having a good product is better delivered with a good customer support. He truly cares about his customers and potential customers. I gave him few suggestions during our conversations and he did take actions. For example, I have suggested that his coursework should have a table of contents. A syllabus is now available.

I do encourage customers to give feedback, suggestions, bug reports and so forth to companies they are dealing with. Sometimes those companies need the feedback to improve. It does not mean every company should do everything a customer ask for since some requests are simply impossible. This is always a healthy sign when a company takes suggestions into account.

Summary: Pros & Cons


Pros
  • Quality content
  • Good investigative tool
  • Excellent customer support
  • Affordable
Cons
  • Cannot manage keywords and descriptions
  • No natural language processing engine
  • User interface and usability issues
  • Too specific dictionary
  • Outdated technology

Conclusion
Statement Analyzer provides a great insight into statement analysis and becomes an interesting investigative tool. Being able to manage keywords and descriptions will make the software useful to many more people than in law enforcement community. Those who would like to learn statement analysis but cannot afford Mark McClish's Statement Analysis Online Training or other seminars, will certainly enjoy a low cost software.

It is still not a pocket version of Mark McClish and there is a lot of work to get there. Many shortcomings mentioned in this review can be corrected in the upcoming months, except perhaps some desired natural language processing features. The future of this software is bright and shiny since its foundations are solid and Mark McClish will certainly continue to provide quality content.

I recommend Mark McClish's Statement Analyzer software.

No comments:

Post a Comment