Testing and assessment methodologies

Overview

DotSec specialises in testing applications and services for its online retail, government, finance and banking, legal, investment, online gaming, education, online payments, insurance, telco and data centre clients.

At DotSec, we pride ourselves on our independence, and on our ability to bring to focus the skills of experts who do not just test and assess systems, but who have developed, integrated and maintained information systems for nearly 20 years. When it comes to assessment and testing, DotSec works with you to understand your business processes, identify your assets, and assess and then manage your risks. You can be certain of receiving a complete and concise report since our assessments are not clouded by any partner, reseller or vendor relationships.

Methodology and tools

DotSec security professionals conduct a wide range of Security Audits and Threat and Risk Assessments (TRAs) which can be uninformed (blind) or informed, and which can include Penetration Tests (pen tests), code reviews and design reviews. Our core process is consistent and is based on a number of standards, primarily AS/NZS ISO 31000:2009, AS ISO/IEC 27001, the Australian Government Information Security Manual (ISM) and IS18/IT&T-14 (State). We are also highly experienced at performing internal and external Cardholder Data Environment (CDE) penetration tests, in line with requirement 11.3 of the PCI DSS.  Of course, most customers will have some unique requirements (generally with either scoping and availability of scoping information, or custom reporting requirements) and we are of course very happy to accommodate those needs.

Whatever the case, our customers are always presented with a detailed report which includes the following sections:

  • Executive summary, which includes summaries of the target of assessment, key findings and key recommendations.

     

  • Scope and asset list, which describes the target(s) of assessment in detail.
  • Findings and recommendations, which includes a list of discovered vulnerabilities, the risk associated with each vulnerability, and a summary of related risk mitigation recommendations.
  • Threat and risk assessment, which include detailed descriptions of the vulnerabilities or short-comings that were discovered, the techniques that were used in the discovery, and a description of how the vulnerability could be exploited in a successful attack.
  • Recommendations, which describes how the level of risk associated with each vulnerability may be reduced to an acceptable level.

We are often asked what tools we use when completing an assessment. The fact of the matter is that any tool is only as good as its owner, and an assessment that is based on the use of a particular tool will always fall short. For the record, tools that we have used in the past include nmap, wireshark, Nessus, various Retina products, Microsoft policy tools, Splunk, various proxies, most command-line network utilities, airsnort, most automated hashing tools  and so on. However, it is our assessors’ experience and insight, not the tools, which allow us to consistently deliver high-quality and valuable results.

Informed or blind?

There are a range of alternatives that you can consider when deciding on the kind of assessment that best suits your goals. Keep in mind of course that these broad categories are not set in stone, and that it may suit you to undertake an assessment which includes elements of each of the categories described below.

Uninformed (blind)

Uninformed or blind assessments are assessments for which the assessor has no information about the target of assessment, other than (in most cases) its location. Blind assessments are somewhat limited because they are generally conducted within a fixed time-frame. An uninformed assessor may not discover all vulnerabilities within that fixed time frame, whereas an attacker with more time on their hands may in fact find it at a later date.

Some clients prefer blind assessments because they require little preparation and have the appearance of being done from the perspective of an Internet-based attacker. This is fine, but it is worth keeping in mind that just because an uninformed assessor does not find a vulnerability in n days, that does not mean that a real attacker will not discover it in n+1 days.

Informed

Informed assessments are assessments for which the assessor understands the details of the target of assessment, and for which the project takes the form of an audit. The assessor will generally begin an informed assessment by reviewing design, policy, and/or as-built implementation documentation. In addition, the assessor will generally have access to the target; the access may be privileged (root, admin, etc.) or unprivileged (for example, a general user of a web application). The exact number and kinds of information, accounts and other information that can be used during an informed assessment will depend upon the scope and aims of the assessment (remember we noted above that many clients have specific goals and requirements).

 

Informed assessments may offer better value for money than uninformed assessments, since the reconnaissance and vulnerability discovery phases of the assessment should be much more effective and efficient than is the case in an uninformed assessment. This is because the assessor can spend time reviewing the correctness and completeness of design and implementation documentation, and clarifying details with the client.

Fit for purpose

Assessments which include elements from the above core groups include source-code assessment (perhaps the most informed assessment of all), audits (where policies and procedures are evaluated) and others. Each client has their own goals and requirements. The key is to a successful assessment is early agreement and documentation of the scope, cost, goals, methodology and desired outcomes.