Wednesday, December 23, 2009

What is hardware in loop simulation?

A system in which hardware and software are both involved satisfying the intended functionality is called as embedded system. This kind of systems must interact with the outer world and hence the outer environment has profound impact on the correct working of the system.

But one cannot have the outer world ready at the time of development or testing of the system which system is going to be part of or it is not possible or advisable to develop or test such system in the real outer world in to gain confidence in the system. Consider an example of a navigation system for launch vehicles; we cannot afford to test them with the real launch vehicles which cost in billions. Take another one, a system which will control a nuclear power plant, can we risk lives if system doesn’t perform correctly in the real plant. Then, how to test these systems?

There is one method known as “Hardware in loop simulation”.
We develop the system in the host environment (normal PC) and burn (put it into) PROM and make the system ready. Now we find out what are the elements in the outer world, the system is going to interact with and how there responses would be if system sends them a particular signal at the time of operation. Create a model out of the outer environment and simulate the outer world in the model. Test the system with that simulated environment. So you don’t have real hardware in picture still u are testing their responses to the system under test.
There is another advantage this testing can be done while system is still under development so that to fine tune the expectations to be implemented in the system.
You can test the control units with extreme conditions that might not be feasible in the real world.

Wednesday, October 28, 2009

Regression testing

Regression testing is process of finding defects in features of software which were working perfectly prior to bug fixation or inclusion of new features. So most of the test cases will be already present and we have to just re-run them. If new feature has been added, we might have to create new test cases.
Why we need regression testing? The answer for the question lies in the design process. However perfectly we design our system or software, there may be some interdependency among various functions and modules of the software called as coupling. As a result, a small change in one function can have catastrophic effect on the other. So to be assured that, any change done to fix a bug in one module has not affected any other module, we use regression testing.

Qualifying a component for regression testing: When any build comes for regression testing, we have to check whether or not the build worth testing. It may be possible that the build doesn’t satisfy the major requirements, so there is not point putting efforts in testing various smaller requirements. So it is advised that always test critical and difficult parts of the software at the first glance and then move on to the trivial and smaller parts. This process of qualifying software for detailed testing is called as smoke testing.

These are the four types of tests one shall run on the software before qualifying it for further testing.
1) Customer based tests: Find out all scenarios in which software is going to be used and run test cases to test software functionalities in those scenarios. Think at higher level i.e. product level and ask what are the test case critical for your customer and run them on priority. Operation profile of the software may be handy
2) Complex tests: Find out which are the test cases which will test majority of the important features of the software.
3) Expected failure tests: From the past experiences and intuition one can find out what are the ways in which the system can fail and try to test those things first before moving to other features.
4) Big picture test: Test the performance of the software and any other quality attributes. If software doesn’t satisfy these attributes tests then avoid testing other features.
Hence one can now understand that regression testing in not about blindly running already created test cases but there are some other subtle issues to be taken care of.

Wednesday, October 7, 2009

Make code readable and understandable: Naming conventions

Uniform naming conventions shall be followed for variables and functions names throughout the code. One can decide upon the naming notations available (Hungarian, Camel, and Pascal etc) as per the requirement and comfortableness but once chosen, notation shall be followed throughout the code.

a) Names shall clearly state the intent of the variable or function
. The most important consideration in naming a variable is that the name fully and accurately describes the entity the variable represents.
For example variable that current payload status is better named as curr_payload_status than cps.

b) Names should be as specific as possibl
e. For example, one can use variable name date to represent current or today’s date. It’s almost a good names but it can represent any date and not the specific today’s date. It can be better named as currentDate.

c) Names should represent problem domain than the solution. They must represent what then how. For example, if one is dealing with temperature obtained by sensor, one shall name it as sensor_temp rather than obtain_temp.

d) Variable should be of Optimum length neither too long nor too short.
For example, for representing no of sensors in payload, numberofsensorsinpayload is too long where as n is too short. Optimal length name will be num_sensors

e) If variable stores result of some computation like total, average, maximum or minimum then names shall be prefixed or suffixed (one shall be followed throughout the code) with these quantifiers.

For example to represent average of all sensor data, average_sensor_data would be good name as compared to avg or avg_data etc.

f) Loop indexes should be given more meaningful name
rather than just i, j, k if loop indexes are to be used outside the loop or loop body is longer than a few lines.

g) For naming any status variable, better name than flag should be given.
For clarity, they must be assigned values and compared to enumerators and global variables which are defined as named constants.
characterType= =CONTROL_CHARACTER is more meaningful than statuflag==0X80

h) Temp should be avoided as much as possible. Instead of that a better name should be given to temporary variable which states the intent of the variable.
Temp= sqrt(b^2-4*a*c)
root[0]= (-b +temp)/2*a
root[1]= (-b -temp)/2*a
It is not advisable to use temp to store the result of expression as it is used in two places after that. Better name and code would be like
Discrimant= sqrt(b^2-4*a*c)
root[0]= (-b + Discrimant)/2*a
root[1]= (-b - Discrimant)/2*a

i) Boolean variable shall be named as done, error, found, success or OK
. One can use them as isDone, isError, isFound, isSuccess etc.

j) When using an enumerated type, you can ensure that it’s clear that members of the type all belong to the same group by using group suffix.

Like Enum color
{
color_Red;
color_Green;
color_Blue;
}

k) When naming constants, name the abstract entity the constant represent rather than the number the constant refers to.
NINE is a bad constant name for no of channels While NO_OF_CHANNELS is a better one. As no of channels can vary, they can be changed at one place only.
Avoid

a) Avoid names with similar meanings like input and inputValue, numRecord and recordNum etc.
b) Avoid variables with different meaning and similar name.
c) Avoid names that sound similar such as wrap and rap.
d) Avoid numerals in names like file1, temp1, temp2 etc.
e) Don’t differentiate variable names solely by capitalization like naming two variables as File and file is a bad practice.

Sunday, September 27, 2009

What is LCSAJ?

LCSAJ stands for Linear Code Sequence and Jump. Typically it contains three parts:
1) Initial step of the program code or a step to which control may jump and continue to execute after that.
2) The program terminating step or an instruction which lead to jump.
3) And the destination step of the jump.

Example:
Let’s there is a program which finds whether input number is prime or not.

1) Printf( “Enter the number:”);
2) Scanf(“%d”,&num) ;
3) bPrime=true;
4) for(int i=2;i&5) {
6) if( num%i==0)
{
7) Printf( “ %d is factor of %d”, i, num);
bPrime = false;
}
8) }
9) if(bPrime==true)
10) {
11) Printf( “Number is Prime”);
12) }
13) End of program.

To find all LCSAJs in code, find all jumps in the program. Those are: (4->9, 8->5, 6->8 and 9->13). All those instructions, to which control could jump from instruction other than the preceding one, are called as start of the LCSAJ.
In the above code starts of LCSAJ are: (1 (entry of the program), 5, 8 and 13).

Now find LCSAJ from each LCSAJ start. For LCSAJ starting at 1, control goes from instruction 1 to 3 and checks for condition at step 4. If the condition is false, then code jump occurs giving first LCSAJ, assuming all other conditions following it are true, (Initial step, branching step and destination step) (1, 4, 9). Now, the case where condition at step 4 is true, control flows from step 5 and then step 6, which is a condition statement. If that condition is false, a new LCSAJ would be formed due to jump which is (1, 6, 8). Similarly, LCSAJs (1, 8, 5) and (1, 9, 13) are also there.
Similarly, LCSAJs could be formed with every start of the LCSAJ.

100% LCSAJ Coverage implies 100% Decision Coverage. But there is a possibility of LCSAJs being infeasible while testing and also effort required for LCSAJ testing and coverage is more as compared to decision coverage.

Thursday, September 24, 2009

IEEE 12207.0 Software life cycle processes

IEEE 12207.0 is a standard which describes the various life cycle processes, activities and tasks to followed in various phases and parts of software life cycle. It is the standard which describes all activities and tasks for software development form cradle to grave from acquisition process to maintenance process along with some umbrella process like management process, infrastructure and training process.
It classifies all process into three parts: Primary processes, supporting processes and organizational process. Diagram below shows the structure of the standard.

Primary processes: these are processes which are performed by primary parties (which are involved in development, operation and maintenance of the software).
These primary parties are: acquirer who acquires the software, Supplier who supplies the software, Developer who develops the software, Operator who operates the software, and maintainer who maintains the software.

1) Acquire process: This process is collection of activities and tasks of the organization which acquires the system, software or service.
2) Supply process: This process is collection of activities and tasks of the organization which supplies the system, software or service.
3) Development process: This process is collection of activities and tasks of the organization which develops the system, software or service.
4) Operation process: This process is collection of activities and tasks of the organization which operates the system, software or service in its live environment.
5) Maintenance process: This process is collection of activities and tasks of the organization which maintains, track changes and keep running the system, software or service in its live environment.

Supporting processes: A supporting process supports another process (primary) in an integral part, as when needed, for successful completion of that process and quality outcome. There are eight supporting processes
1) Documentation process: It defines and elaborates the activities and tasks to be performed while recording the information of life cycle process; it can be requirements, design or any contract of the process.
2) Configuration management process: It defines all activities and tasks which are required for proper configuration management of project deliverable and artifacts.
3) Quality assurance process: It defines and elaborates various activities and tasks to assure quality of the product and process and ensure that product is developed as per requirements and adhere to their established plans.
4) Verification process: It defines activities (for acquirer, supplier, or independent party depending on the nature and time of conduction of process) for proper verification of the product being developed in varying depth based on the nature of the software.
5) Validation process: It defines activities (for acquirer, supplier, or independent party depending on the nature and time of conduction of process) for proper validation of the product being developed in varying depth based on the nature of the software.
Important difference between verification and validation is that verification is done for both process and product but validation is of product only.
Verification ensures that we are making system right.
Validation ensures that we have made the right system.
6) Joint review process: It defines all activities for reviewing the intermediate and final product and process. There are two parties involved in the process one is who is reviewing (may be independent) other being reviewed.
7) Audit process: It defines all activities and tasks which are essential for determining whether the process followed in development is compliance to the standard adopted. Here also there are two parties involved auditing and other being audited.
8) Problem resolution process: Defines activities and tasks for analyzing and removing the problems (including nonconformance), whatever their nature or source, that are discovered during the execution of development, operation, maintenance, or other processes.

3) Organizational processes: These processes are employed by organization for establishing and developing the underlying structure for life cycle process and producing personnel for improving the process and structure. These are umbrella process and are outside the project limits and can be employed and required across the projects. There are four organizational processes:
1) Management process: It defines activities and tasks for management for maintaining and improving the process including project management and execution of life cycle process.
2) Infrastructure process: It defines basic activities for establishing underlying structure for life cycle processes.
3) Improvement process: It defines basic activities and tasks (for all primary and supporting parties) for establish, measure, control and improve life cycle process.
4) Training process: It defines activities and tasks to produce skilled and adequately trained manpower for life cycle processes.
These are various processes in 12207. Software life cycle processes standard. We will discuss each and every process in detail in forth coming posts.

Tuesday, September 22, 2009

Unit testing techniques

Unit is an atomic part of a program which could not be decomposed in smaller parts. Testing that part for its correct implementation is called unit testing. There are various techniques for conducting unit testing. These techniques are complementary to each other and find different set of errors when applied to unit, one after the other.

1) Specification based testing techniques:
This technique generates test cases based on program's behavior,input domain and expected output.Following are specification based techniques
a) Equivalence partition: Under the technique, input domain of the unit is partitioned into various classes. Each class contains input values which would have similar effect on the code. Test cases select input values at random,from these classes.

b) Boundary value analysis: test cases are generated so that they can cover the boundaries of any condition in the code. Generally, it is performed after equivalence partition. One test case is designed for inside of the classes and as many as necessary to cover its limits.

c) Decision tables and cause effect graphing: Inputs for the unit can occur in any combinations. Those combinations are represented and analyzed with decision tables and cause effect graphing. Then test cases are generated for all the possible input output combinations.

d) Random testing: Test cases are generated at random according to the input domain and expected output domain specified in specifications.

2) Code based testing techniques: These testing techniques can be again classified into two categories:
Control flow based criteria: This varies as to how well and what part of code, the test case covers in the program control flow.
Data flow based criteria: In this, test cases cover the execution space between variable defined and where they are used in the program flow.

Let’s consider the control flow based criteria techniques:
a) Statement coverage: Test cases generated for this criterion must ensure that they execute each and every statement of the code.

b) Decision (branch) coverage: Code contains many decisions and corresponding branches for those decisions. To fulfill the decision or branch coverage criterion test cases must ensure that all the program decisions take the value true or false and execute the corresponding code

c) Condition (Predicate) Coverage: In practice, decisions compose of many conditions. Combination of values attained by these conditions decides the final outcome for decisions. To satisfy condition coverage criterion, test cases must ensure that all conditions present in the decision take the value true and false.

d) Modified condition/ decision coverage: There may be a case where we have tested the condition and decision in the program, but still individual effect of each condition on the decision is not tested. There may be cases of short circuiting in logical expressions. To test the independent influence of every condition on decision modified condition/ decision coverage is used.

Monday, September 21, 2009

Nature of bugs found in code inspection

Code inspection is an important process in any mission or safety critical system development. It’s a type of static testing in which code is not executed but critically analyzed in order to find some subtle bugs which are not detectable or require lot of efforts to detect in testing phase. The point of interest is what kind of defects one can find in code inspection. This article analyses bugs classes found in the code in code inspection process.

1) Maintainability of the code: No code is perfect when it has been written. It is bound to change, enhanced and modified for adaptation to different platforms. So it is necessary that code written is understandable and maintainable. Code inspection focuses on this goal. Under this, following points are looked for in code:
a) Code to comment ratio is proper and whether comments explain intent of the code appropriately and correctly.
b) Whether all the variable names and function names are meaningful and clearly state the intention.
c) Whether the coding standards or guidelines are adhered to.
d) Whether code has proper logical flow or not?

2) Design flaws: Code reflects design of the software. What if the deign itself contains flaws. Since software is tested against requirements and design, it is very difficult to detect these kind of defects in testing phase. In that case, code inspection provides an additional design review step. There are issues which will never fail the system but very dangerous from the quality point of view. For example, when designing system, one designs a structure to keep track of some of the hardware channels status like if they were empty or full etc. The very thought instrumental behind the design was that structure could be used bit wise and memory could be saved which would have not possible by using arrays. But this thought didn’t consider that system had some functionality which requires checking status of all channels at the same time like for averaging of all channels data. Since the data structure used was bit wise structure, one has to check all the fields of the structure one by one resulting in very lengthy and unstructured code which would have been very simple, in single loop if array has been used.

3) Structural flaws: Proper structure of the code is must for understanding and testing of the software. Following points which are given stress on in code inspection process:

a) Does every function implement single functionality and is testable?
b) Does length of the function justifiable or shall it be broken into smaller ones?
c) Is any dead or extra code present in the code?
d) Is any unreachable code present in the code?
e) Whether there are multiple exists for functions and can they be removed?
f) Does every variable, that has been assigned value in any conditional block, get initialized prior to usage?

4) Logical errors: These are errors which are infused in the coding phase by developer because of ambiguous design or unclear requirements. As there is no guard against them in requirements and design, these kinds of bugs are subtle,not obvious in nature,may occur once in many execution cycles or when a particular sequence of events occur.
For example, lets say, software works in sequence of steps like power on, operation on, check of automatic corrections,operation off and then power off. In operation phase, it first checks whether given number of passes have already been elapsed in operation then only does the automatic correction. The variable which keeps track of no of passes was reinitialized at the power on stage. Assuming that software goes to operation off state; again comes to operation on state without going to power off state. Now since this is a fresh operation, automatic correction shall be applied after given no of passes have been elapsed in this operation, but as variable keeping track of no of passes is not reinitialized hence contains last value, so software does not wait for given no of passes to elapse and starts performing automatic correction. This error will not hamper the functioning of the software but will surely hamper the algorithm which calculates the corrections to be applied to the data. These kinds of errors are found in the code inspection by hypothesis.

There is subtle difference between code walkthrough and code inspection
Code walkthrough is an informal process which is conducted by developer team. Peer reviews are kind of code walkthrough.
Code inspection is more formal process with formal team,roles and meetings. Code inspections report is asked for in project and process audits. Checklists are used for code inspection.