• Verification: Check whether we are building the product right
• Validation: Whether we are building the right product

Some authors propose several strategies to improve software testing results:

• Quantify Requirements long before the testing begins: So testers can measure how close the product is to the client’s idea.
• Specify testing goals: Time between failures, cost of fixing a failure, etc
• Understand the client’s behaviour and develop a profile:  Focus software testing on its real usage
• Testing plan featuring ‘Quick Testing Cycles’
• Self-ControlProduct: Failure detection
• Formal Testing Techniques: (Alright, they might be cool but too expensive)
• Continuous Integration

 Strategies for Testing Software

Unit Testing:

• Verification of each module to check whether it implements the contract it relays on.
• Since some other components it depends on might not be yet available, some interface simulation must be developed.

Integration Testing:

• The goal of this process is to test the interaction between modules and prevent any problema when interconecting interfaces
• Top-Botton Aproach needs to generate simulators for no yet available modules
• Bottom-Up Approach tests first low-level modules and keeps adding higher-level ones so no simulation is needed
• Regression Tests are needed to be sure that when some modules has been modified all other keeps working ok
• Smoke Testing is a lightweight tests which aims to detect problem in a early stage. It must be tested daily

Validation

• Focuses to check whether the product stays close to the requirements
• Uses Alpha (developer + user) and Beta (only user) tests

Introduction to the PSP

The book is written in a pretty informal way, the author seems to talk to his son or someone really close, which I don’t really mind but, from my point of view, adds extra literature not necessary at all to explain the PSP.

The author focuses in two aspects along the book, measuring / planning and defects. The most valuable part and which I found more interesting is the one dealing with personal planning since it can help you not only when dealing with software projects but in your real life.

I really think that applying techniques such as registering tasks durations, interruptions management and creating week summary can help you become more productive and eliminate those periods of the day you think you are working but your not.

I also agree with the author that using his techniques you can become more realistic when planning a new project I think it is not because of the whole planning stuff but you can know exactly how many time are you really working in your projects without all that bloat of interruptions.

The part I don’t really like is the second half of the book which focuses in dealing with defects. The author ideas are only applicable when dealing with beginners defects and I don’t find useful any of the metrics he proposes.

In few words, this books is worth reading if you feel you are wasting your time and want to become more productive and/or want to make sure you can make realistic project plans.

• A design must present an architecture built using known pattern designs, components with the right characteristics and that can be implemented in an incremental way.
• Must be modular, divided in subsystems
• Must lead to interfaces which reduce the complexity when connecting components
• Must be generated from a reproducible method and the information produced during the analysis
• Must use notation to express its meaning correctly

COQ takes into account the following costs:

• Failure costs: All costs of fixing a defect e.g. Patching a deployed product, using the debugger, reviewing code, etc
• Appraisal costs: Work done evaluating whether a product has defects, excluding time spend on fixing them.
• Prevention costs: Resources used to improve the process to reduce the number of defects.

Following I will describe the PSP approximation to COQ a simplified way of calculating this measurement. The author states than this version of COQ shows the same effectiveness as the long COQ version.

• Failure costs: all compiling time, all testing time.
• Appraisal cost: All review time

Cost of Quality is calculated as a percentage of total development time:

• Appraisal COQ: Sum of all review time as percentage of total development time.
• Failure COQ: Sum of all compile and test time as percentage of total development time.

Product: Something produced for a company or client

Project: Used to produce a Product

Task: Element of work

Process: Way to do projects

Phase: Steps proposed in a process (e.g. Planning, development, testing). Each phase is composed of tasks

• Assume synchronous system with crashing the only fail
• Replica Managers contain the replica in a computer and perform operations upon them
• Front-end issues request to one or more replica managers
• Coordination: Replica managers coordinate the requests FIFO, Casual, Total
• Replica manager pass back the first result for responsiveness or what majority has reply for byzantine failures

Group Communication:

Group Membership Service:

1. Interface for group membership changes
2. Failure detector: To track failure of members
3. Notifying of membership changes:
4. Group address expansion: change address from single to all members

IP multicast has all except 2 and 3.

Group views: Ordered lists of current group members.

A member can be exclude if it is suspected even it has not crashed.

View Delivery:

• Instead of query every member views of the group are delivery to applications
• They follow Order, Integrity and Non-triviality (if two processes join they can communicate and if they split in two groups that should be reflected)

View-Synchronous Group Communication:

• Agreement: Same set of message delivered in any given view
• Integrity: A message is delivered just once
• Validity: If a process fails a new view is immediately send.

Fault-Tolerant Services

Linearizability:

• Interleaved sequence meets the spec of a correct copy of the objects
• Order of operations is consistent with the real times at which they occurred

Sequential Consistency:

• Interleaved sequence meets the spec of a correct copy of the objects
• Order of operations is consistent with the program order where those where executed

Passive (primary-backup) Replication

• The primary is replace by one backup, otherwise it would have been incorrect
• Replica managers agree on which operations had been performed when secondary takes over
• Can be used even in a non-deterministic way, multi-threaded apps
• Requires f + 1 replica managers

Active Replication

• Each request is sent to all the servers and executed simultaneously
• This schema achieve sequenciality
• Total order in which the replica servers process request is not the same as the time order in which the client requested
• 2f+1 replica managers for byzantine failures

High-Available Services

The active behaviour of waiting for all the replica managers to reply undesirable for highly available operations

Gosssip Architecture:

Two basic operations, queries (RO) and updates (modify without reading)

The system guarantees the following, even if the replica managers can communicate with each other

• Each client obtains consistent service over time: Replica managers provide data reflecting at least the updates observed by that client so far
• Relaxed consistency between replicas: All replica managers received all the updates and apply them in an order to suit the application needs

Supports causal update ordering:

• Immediate ordering: Applied causal and forced ordering are no happened-before related
• Causal ordering: Posting items, they can be ordered by subject: Oranges, “Re: Oranges”
• Forced-ordering: members joining in the same order
• Immediate-ordering: deleting a member, so a user can not being active for some late replica

Outline of a gossip service:

1. Request: Front-end sends to only a single replica manager. Block in query, return asap in updates but client may be held till update has been sent to f+1 replica managers to avoid f failures
2. Update Response: Replica managers replies as soon as received an update
3. Coordination: A request is held to apply the ordering, maybe waiting for other replica manager updates.
4. Execution
5. Query Response
6. Agreement: Replica managers update one another by gossip messages. Messages exchanged occasionally, lazy fashion or when a replica managers finds out that it is missing an update to send.

Front end’s timestamp:

• Front-ends keep their own vector timestamp passing it to replica manager and merging when they receive it back
• They pass the timestamp to clients

Replica Manager State:

• Value: Each Replica Manager is a state machine and changes applying updates operations
• Value Timestamp: Vector timestamp that represents updates reflected in the value. One entry for every replica manager.
• Update Log: 1. It can not apply an update yet 2. after being applied it has been in other replica managers
• Replica Timestamp: vector timestamp representing accepted updates. Maybe different from value timestamp
• Execute Operation Table: Preventing updates applied when coming from Front end and other replica managers, it uses front-end unique identifiers.
• Timestamp Table: Timestamps from other replica managers received in gossip messages, used to know when updates have been applied to all other.
• Each position of the vector timestamp are updates received from the ith replica manager.

Query Operations:

• The query operation contains a timestamp reflecting the latest version of the value the front-end has read/updated.
• The replica managers must return a value, at least, as recent as the timestamp indicates

Updates operations in causal order:

The front-end assigns an identifier and adds the timestamp

The replica managers checks whether the id has been previously processed, if not it updates the ith position of the vector timestamp and send it back to the front-end.

The front-end merges the timestamp

Forced and immediate update operations:

• Forced: Append identifier to timestamp and process them in that order by a Primary Replica Manger
• Immediate: Primary server synchronize with other replica managers to reach the order.

Gossip Messages:

• Contains the log and the replica timestamp
• Merge arriving log with its own log
• Apply stable updates (and no executed before)
• Eliminate records from log when they have been applied everywhere

Update Propagation:

• Random: Replica managers randomly choose a partner to synch
• Topological: Replica managers arranged in a graph
• Tuned to the application

Discussion:

• Lazy approach makes gossip inappropriate for near real-time requirements
• Bad scalability, as the number of replica managers grows so does the number of gossips.
• Making some replica managers RO may improve scalability

Bayou and the Operational Transformation Approach:

• Replica Managers exchange updates in pairs
• Users make any updates, when replica managers merge the updates and resolve conflicts (e.g. boss appointments first)
• Updates marked as tentative when they can be rearranged. They move to committed.
• Before update operation, replica managers perform dependency checks if indicates a conflict a merge is applied
• Replication is non-transparent to the application

Coda Filesystem:

• Focus on High Availability despite disconnected operation
• Replication began to be a problem in AFS, for widely accessed shared files
• Provides users with share file repository and allow the to rely on local resources when repository non-accsible.
• Relies on replication of file volumes to achieve hight throughput
• Vice process, the same as Replica Managers
• Venus, mix of Front end and Replica managers
• VSG, Volume Storage Group, servers holding replicas of a file volume.
• Changes notified to clients via callback and on close, modified files are broadcast in parallel to all the servers.
• Code client caches all the files
• CVV, Coda Version Vector, vector timestamp each entry is the updates on each server
• An open in code provides the most recent copy of the file from the current AVSG, if all servers down, a cached copy is used
• Venus processes (client) must check for AVSG shrink/growth
• Performance decreases as more users and fold replication enabled

• Processes connected by reliable channels
• Only possible file is a crash
• Systems Asynchronous

Distributed Mutual Exclusion

If server involved allowing access to one process at a time

Algorithms evaluated following:

• Bandwidth consumed
• Client delay
• Throughput of the system

Central Server Algorithm:

• A process sends a request to the server
• Server gives Token
• When queue is empty next one is chosen
• Entering costs two messages (request then grant)
• Exiting one message
• The server may become a bottleneck

A ring-based algorithm:

• Server-less approach
• Connection between one process and the following
• One a process leaves the critical section sends the token away to the next one
• Continuously consumes bandwidth
• Entering Delay between 0 to N
• Leaving 0 messages

Ricart and Agrawala’s Algorithm:

• Mutual exclusion between N peer processes using multicast
• Entering: send multicast message and enters when replied by all other processes
• Communication between processes and each one has Lamport clock
• Entering messages <T, pi> T timestamp and pi process’ id.
• Process’ state: RELEASED, WANTED, HELD
• If all process are in RELEASED reply immediately, if any is in HELD waits
• Concurrent requests ordered by T and then by pi
• Entering 2(N – 1) or 1 when hardware support for muticast
• Synchronization delay only 1 message

Maekawa’s Voting Algorithm:

• Subsets of all processess
• Subsets share at least one process
• Entering: send message to your subset and wait for all to reply
• A process reply only if has not HELD state and has not replied to other process already
• Leaving: Process send exiting message and others pop next one from the queue
• Dead-lock prone
• Bandwidth: Enterying 2*sqrt(N), Leaving sqrt(N)

Process Election:

Measurement

• Bandwidth utilization
• Turnaround Time

Ring-based Algorithm:

• Initially all non-participants, anyone can begin one
• Process marks itself as participant and sends election message to its neighbour
• Receiver compares identifier. If greater, forwards msg, if not and it is not a participant, substitutes the identifier and forwardit. But do not forward any message if it is already a participant
• If the received message is the same identifier, becomes coordinator and sends elected to neighbour
• When elected process marks itself as non-participant

Bully Algorithm:

• Allows process to crash during election, but message delivery reliable
• Assumes synchronous system
• Each process knows which processes have higher identifiers
• Process begins election when, through timeouts, finds coordinator has died
• If the process has the highest id elects itself as coordinator, sending coordinator message
• Other processes begin election sending an election message to those with higher identifiers and waits answer, if timeout elects itself as coordinator and announces to lower ids
• Receiver of election begins another election
• Two processes may announce being coordinators at the same time
• O(N^2) in the worst case or N – 2 in the best

Verification: Does it meet the requirements?

Validation: Does it fit its purpose?

And the corresponding entry in Wikipedia.