Database Testing

Why test the database?

The database persists data that your application (and organization) depends on. The data thus persisted is most often of mission-critical nature and a key asset for the organization. Also, many of today's data-enabled applications implement a fair amount of their functionality and business logic in the database itself. For an enterprise class application, the data in the database would be accessed and updated (insert/modify/delete) simultaneously by a large number of users (think thousands to millions depending on the scale of your application's usage).

The above statements highlight a few areas where database testing is needed. One, to validate the quality of data being persisted. Two, if we plan to test the application code, it is imperative that we also test the code in the database which implements the business functionality and three, we should plan for non-functional database testing to support the usage of the database in a real-world deployment scenario.

Let us briefly look at the above mentioned database test areas.

1. Data quality testing

Testing the quality of the data may be approached in three ways - data validity testing, data integrity testing and data format testing.

a) Data validity testing - is done to verify the validity of the data that is stored in the database. When data is entered via the front end application, check if the data is correctly updated in the back-end database. Apart from the positive checks, look for other behavior such as data truncation, verify how null/empty field values are handled, verify how special characters or code snippets are handled in the database. Check that the right columns in the right tables are being updated. Data validity testing normally involves use of SQL queries to validate the data.

b) Data integrity testing - involves testing referential integrity and application of constraints (foreign/primary key). When a data field is subject to modification (insert, update or delete), the database should be verified for appropriate changes to related entities such as primary key/foreign key relationships and that referential integrity of the data is maintained

c) Data format testing - involves verifying the size and type of fields that store data in the database with those that accept data in the application. This can help identify mismatches between the type or size of data that is accepted by the front-end vs what the database can store. Example: the application may accept text data but try to store in a numeric or date field in the database or else the application may accept data of greater length than the max length for the corresponding field in the database. This may not throw errors during routine application usage but may store incorrect or erroneous data in the database which could have repercussions elsewhere or at a later stage
   
2. Database code testing - involves testing the code in the database which implements business logic and functionality. Examples of such code include, stored procedures, views (read-only/updateable) and event driven items such as triggers. Each stored procedure is tested distinctly for its functionality. When a stored procedure implements multiple functions, each function is tested separately. Stored procedure testing would look at testing the arguments that are passed to the stored procedure in terms of the number, type and order of arguments plus the return value. Both positive and negative tests can be devised to test stored procedures. Views both read only and updateable are tested either as stored queries that dynamically retrieve values from the database and/or allow updates to the database. In case where updates are allowed, data validity and integrity testing is done. Event driven items are tested by verifying the events that could trigger actions and the actions themselves for functional correctness

3. Non-functional database testing

In most real-world mission critical deployments, to ensure database scalability, security, availability and recover-ability with minimal/no-loss of data, it is important to test the following non-functional areas

a) Database performance testing (load/stress/longevity/scalability)
b) Database security testing
c) Database replication testing
d) Database fail-over testing
e) Database recovery testing

This sums up in brief, the subject of database testing.

Scrum, an Agile development methodology

What is Scrum and what is the relationship between Scrum and Agile?

Scrum is an iterative and incremental Agile development methodology. Scrum may be viewed as an agile framework for developing software. Unlike many other software development methodologies, scrum does not provide a complete template or detailed description of what to do during software development. Scrum prescribes desired outcomes and leaves it to the agile scrum team to best determine how to solve the problems they encounter. Scrum may be used for both software development and software maintenance projects.

The term "scrum" is borrowed from an analogy put forth in a 1986 study by Takeuchi and Nonaka wherein they described a new holistic approach that would increase speed and flexibility in new product development. In that study, Takeuchi and Nonaka compare high-performing, cross-functional teams to the scrum formation used by rugby teams.

Principles of Scrum

These principles have been borrowed from the Agile Manifesto
 

Scrum values -

• Individuals and interactions over processes and tools
• Completed functionality over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, the items on the left matter more in an agile development methodology such as scrum.

Scrum relies on self-organizing and cross-functional teams known as scrum teams. The scrum team is self-organizing in that there is no overall team leader or manager who allocates tasks or specifies how a problem is to be solved. Those are issues that are decided by the scrum team as a whole. The scrum team is cross-functional in that there is representation from all functional groups (developers, testers, technical writers, etc.) so that everyone necessary to produce working and shippable software is involved.

Definitions of key terms used in Scrum

• Product owner - represents the business, customers or users and guides the team toward building the right product.
• ScrumMaster - ensures that the team is functional and productive. The ScrumMaster may be viewed as the coach for the scrum team, helping scrum team members use the scrum framework
• Scrum team - cross-functional and self-organizing team that gets the work done
• Sprint planning - an event/meeting wherein the scrum team meets with the product owner to choose work (set of tasks/features/requirements) to deliver during a sprint
• Daily scrum - an event/meeting wherein the scrum team meets every day during the duration of a sprint to share about progress made, plans and obstacles if any
• Sprint reviews - an event wherein the scrum team demonstrates to the product owner what it has accomplished during the sprint being reviewed
• Sprint retrospectives - an event wherein the scrum team looks at the recent sprint that has completed for ways to improve product development
• Product backlog - it is a dynamic and prioritized list of requirements for the project. The product backlog will have the features and requirements for a product development effort. It is dynamic in that requirements may be added or removed to the list
• Sprint backlog - it is a negotiated subset of the product backlog that the scrum team commits to accomplish during the time frame of a sprint. The items in the sprint backlog are further broken down into detailed tasks. The scrum team works together collaboratively to complete the items in the sprint backlog
• Burndown chart - represents the work remaining for the sprint and the overall project
• Potentially Shippable - it means that the work output delivered at the end of a sprint is in a form/state of completion that it may be released to a customer if required

The Scrum process in brief 

• The product owner creates the product backlog with requirements for the product that is to be developed
• The scrum team meets for the sprint planning meeting and picks up a set of high priority requirements from the product backlog to work on during the sprint. This forms the sprint backlog
• The duration of the sprint is normally between 2-4 weeks. During this time, the scrum team meets daily to assess progress and impediments if any
• No new requirements are added to the sprint backlog once the sprint starts
• The scrum master keeps the team focused on the objectives for the sprint
• During the sprint, the cross-functional scrum team works together to deliver to its commitments
• Towards the end of the sprint, a potentially shippable unit of work has been produced
• The sprint ends with a sprint review and retrospective
• The iterative cycle continues. As the next sprint begins, the scrum team chooses another set of requirements from the product backlog to work on during the new sprint
• The cycle repeats until the project is completed i.e. either the items in the product backlog have been delivered or the time line for release is reached or budget exhausted. Either way, when the project ends, Scrum would ensure that the higher priority and important requirements have been addressed

I have been working with scrum teams for a while now after having completed a scrum master certification program. Our groups transitioned to scrum from a traditional software development approach. Scrum is a simple framework that is easy to understand but poses a certain amount of challenge to implement. This is especially true in organizations that have been used to more heavy-weight traditional software development methodologies. It requires understanding and commitment from all levels of the organization to push for change and adopt scrum.

Software Testing notes

"If debugging is the process of removing software bugs, then programming must be the process of putting them in." - Dijkstra
 

It sure is safe to say that software is ubiquitous and touches almost everyone on this planet either directly or otherwise. The software we help produce has the potential for wide reaching impact. A chief determinant of the nature of impact, whether positive or negative, hinges on the quality of the software we produce. Testing directly contributes to improving quality of software by way of detecting defects and enabling these to be addressed before the product ships.

When defects are not detected and consequently left un-addressed, failures result during operation of the software. These failures can have significant cost implications for the producer of the software which may include (and not restricted to) costs to address these issues that are reported back from customers and issue patches, loss of customer confidence and credibility, loss or corruption of data which in turn has potential for causing much damage, legal implications due to failure or non-compliance, and many other repercussions which are best avoided by taking pro-active steps to prevent and eliminate defects during software production. A relatively smaller investment in preventing software defects from shipping out helps to avoid spending a much larger amount later, on handling the consequences of those defects.

So, what is the purpose of testing? We may summarize the purpose of testing into the following three points.

1.    To validate conformance of the software to the business requirements
2.    To verify conformance of the software to the design and specifications
3.    To find errors

An important element in finding errors is timing. The sooner in the development cycle when errors are detected, the less expensive it is to fix them. Studies show that the longer errors remain undetected across the development life cycle, the greater is the cost of fixing these later. Different statistics provide varying estimates of cost involved in fixing defects at different stages of software development, testing and deployment. However, all of them agree that the cost is least for defects identified during the initial requirements stage, increasing thereon for every subsequent stage such as design, implementation, testing and the highest for defects found after the product has shipped. For example, an estimate of the cost of fixing defects post release is said to be over 40 times the cost of fixing them at the requirements stage. 

The point here is that for software testing to be of greater value to the business, testing must not be relegated to the fag end of the development life cycle to come in only post implementation. The earlier you have testing engaged, the greater the defects that may be prevented from being carried over across development phases and lesser the cost to address them.

Software Testing notes

“Program testing can be used to show the presence of bugs, but never to show their absence” - Dijkstra

One of the fundamental principles in software testing is that testing can be used to show the presence of errors, but not their absence. To prove that the software is free of defects would require the system to be tested completely. Complete testing would include tasks such as, testing the system with - every possible input value that it can take, every combination of inputs that are possible to be passed in, every possible path of execution, every possible compatibility scenario, every possible interactions with other components be it software, hardware or human, every combination and version of dependencies, every possible situation in which the system may be used, etc. The entire space of what is possible to test is infinite for a non-trivial software system.

It is not just the number of tests that are infinite, in most cases the number of possible input values themselves could be infinite. Even if you were to consider a very simple input field which accepts just a set of numbers, it would require testing of all the valid numbers that will be accepted as well as all the invalid numbers which are either less than the least or greater than the greatest number that the field is supposed to accept. Similarly, when you have a set of input fields where user data is accepted, every combination of input values both valid and invalid that may be passed to these fields need to be tested for testing to be truly complete. If you thought that testing with such an extremely large set of input values were enough, think again. Additional tests may be added to test for scenarios involving editing or altering of values as they are being entered or delaying entry of values to check for time-out handling and so on.  Even if one were to embark on an attempt to do complete testing, the fact is that software testing is not an isolated function with unlimited time and budget at its disposal. Testers in the real world are required to complete testing in a set amount of time and within budget. Complete testing almost never fits within these boundaries.

Given the fact that on one hand you cannot truly state that there are no defects in the system until you have tested it completely while on the other hand complete testing is not practically possible, it is likely that testing may be viewed as a fundamentally flawed process. While there are an infinite number of potential defects in a software system of non-trivial complexity and size, testing can theoretically only provide an infinitely small level of quantitative confidence in the quality of the software. So, would it be right to state that software testing is not useful ?