cs130-eggert

Computer Science 130 - Software Engineering

http://web.cs.ucla.edu/classes/spring16/cs130

35%    final exam (open book/notes)
15%    midterm (April 27) (open book/notes)
40%    project
2.5%   requirements homework
7.5%   quizzes (closed book)

June 5 drop-dead date, last day of instruction


Paul Eggert background

  student at UCLA
  did not take software engineering because field did not exist
  became faculty for 3 years and got bored
  left university and started company -> crashed in 5 years
  worked for second largest computer company in the world (Unisys, power of^2)
  another startup -> 12 years (< 50 employees)
  UCLA (after that)

  back-and-forth between academia and industry

==============
Mar. 28, 2016
==============

1. real-world-ish project

   build code that somebody really wants and does not have before
   ideally it should have a client with requirements
   interact with client and produce what client wants

   outside clients, on/off campus, they will not know about software
   ------------------------------------------------------------------
   - IBM
   - Aerospace
   - Anderson School of UCLA
   - Psychology Department
   - Calthorp School

   work in teams of 5, 2-3 teams/client
   -------------------------------------
   - cooperating, where two teams work on different parts of project and
     integrate the whole project in the end
   - competing, where different teams take different approaches to the same
     project

  student projects are also welcome
  ----------------------------------

  project open-source only, typically BSD, GNU, Apache, MIT licensed
  -------------------------------------------------------------------

2. engineering process and practice (higher-level)

   how software engineering works
   almost social-science
   how engineers work with/against each other to build something that works

   how this process works, or doesn't work

   instead of dealing with software, deal with the methods and ways of
   developing the software

3. construction (lower-level)

   know program you want to build, what are the methods and best practices for
   building this piece of software

   actual construction details of software
   ----------------------------------------
   - testing & debugging
   - modularization techniques

   may seem obvious but are actually not
   may be confusing and have pitfalls

   learn not to obvious things in 'obvious' things


What is of software and why is it different?
---------------------------------------------
- set of rules a computer follows
  
  microcode in commercial chips is not included as software
    hardwired into the CPU
    cannot be changed at all

  firmware is not software either
    halfway between hardware & software
    code that is tightly bound with hardware
    low-level code + hard to change (ROM)

    we can write very low-level code
    this is software but very low-level code
    calculate a random number

      // x86-64 machine, includes instruction
      // return a random number 0 ~ 2^63 - 1
      // this works only on x86-64 Hasweller machine
      long randnum(void) { asm ("RDRAND"); }


  1. software is easy to change
     ---------------------------
     mutability of instructions
     mutability brings forth engineering problems that make software hard

  2. it's not manufactured in the traditional sense
     -----------------------------------------------
     don't have to worry about manufacturing
     - type 'cp' or 'scp' to copy program and ship

  3. it doesn't wear out
     --------------------
     don't have to worry about parts wearing out
     parts fail -> lubrication/replacement/repair
     can run program any number of times you want

     hardware failure rate (bathtub curve)
       break-in period: not manufactured correctly
       wear out: finally breaks down because of time

     software failure rate (spikey bathtub curve)
       increases failure rate with each new release
       decreases failure rate as bugs get fixed
       failure rate increases overall as software gets bigger

  4. there are no spare parts
     -------------------------
     if program crashes, replacing with same new copy will not fix it
     
     - maybe revert to previous version
       gzip 1.7 on Solaris 10 with Oracle cc x86-64 crashes
         configuration uses match.S  <-- machine code does not run, dumps core
       gzip 1.6 on this platform (can be considered spare part)

     - maybe substitute a different implementation
       use pigz instead
         different implementation of gzip compression algorithm
         runs in parallel


What is software engineering?
------------------------------
1960s: software was out of control
       "software crisis": too many bugs in software
       need discipline for software development

F.L. Baver (1969): the [establishment] and [use] of [sound] [engineering]
                   principles in order to obtain [economically] software that
                   is [reliable] and works [efficiently] on [real machines]

this definition did not include teamwork


Shellshock Bug (reported 2014-09-24)
-------------------------------------
bug in bash

  $ cp = '(){ ... }'
  $ export cp

this treated 'cp' as a function definition
then running 'cp a b' would run the function above
this bug had been in bash for 20 years

San Diego Union (reported 1994-04-27)
--------------------------------------
DMV spent $44 million on software that doesn't work and will never work
nobody was responsible

wanted to convert circa 1960 database to modern relational database
had bottleneck that could not handle the load that the DMV threw at it


* Most software projects fail.


Alternatives to software engineering
-------------------------------------
* add people to a late project
  often not a good idea

* outsourcing
  outsource to another group
  hire contractor, and let somebody else do the job
  sometimes works, but 'distance' will be a problem
  lose control of the software (intellectual rights access)

* vague objectives and requirements constantly changing
  "agile development"
  sometimes works but often doesn't

* fire and forget
  get some code out the door by the stated deadline and never touching again

* code is all there is
  don't bother to write documentation becuase it is a hassle and probably
  doesn't match the code either

* voluminous PPM (Policies & Procedures Manuals)
  software development process that documents everything
  software development process takes much longer than it may take
  good for developing reliable software
  and proving that it is reliable


software engineering           vs.           computer science
----------------------------------------------------------------------
practical problems of                        theory & methods that
 producing software                            underly programs

software engineering           vs.           system engineering
----------------------------------------------------------------------
just software plus                       software, hardware, firmware 
 human interface                                process design
                                                    policy


==============
Mar. 30, 2016
==============

Presentations for projects

1. The Aerospace Corporation (Mustafa Alammar)
   Enterprise Information Systems/AppDev

   Aerocube-7

   Mission Assurance
     making sure rockets launch successfully
     consulting company
     helping SpaceX certify rockets
     launch national satellites

     SpaceX, NRO Satellites, U.S. Space Systems, Cyber Security

   R&D
     space systems
     spectral imaging
     small satellites (micro/pico-satellites)

   VAEROS
     more on civil and commercial customers, NASA and SpaceX
     'container inspection tests at Port of Oakland'
     modeling and simulation
     CI security

   technology used: SQL, PHP, Java, HTML, CSS, JavaScript, AngularJS


 Last year: orbital debris tracking

   Center for Orbital Reentry & Depris Studies (CORDS)
     tracking orbital debris
     visualizing and tracking for all
   
   Junk Explorer
     MySQL
     Bootstrap
     Google Earth/Maps
     www.space-track.org
     Digital Ocean


 This year: cyber security

   http://map.norsecorp.com/

   develop site and server security intrusion detection, monitoring and more
   develop plugins, applications, algorithms, or systems to detect and
     defend against security vulnerabilities on pubilc-facing websites

   Requirements
   
   (1) makes a measurable or testable difference in improving site security
         DDoS mitigation of security vulnerabilities detected or eliminated
         span detection/removal
         success rate of 'hack' removal

   (2) easy to use or install
         if (application.needsConstantAttention == true)
           application.uninstall

   (3) reliable
         if (application.falsePositiveRate > 0.1)
           application.uninstall()       

   (4) leverage existing solutions
    
        APIs
          Akismet
          ReCAPTCHA
          CLOUDFARE
      
        fill in the gaps
          usability
          effectiveness
          ease of installation
          cost
          compatibility
      
        remember the user


2. Connexity

   tech-driven marketing solutions company
   connect reteailers to customers they care about

   used to be called Shopzilla

   Project: Publisher Challenge
   -----------------------------
   use the Shopzilla Catalog API to build own shopping website
   expose shopping content
   develop back-end and front-end components of shopping site
   compete for most redirects and 'revenue'

   Challenges
   -----------
   API integration
   what data should your website use
   determining what additional information would be useful
   completing the flow-API -> backend logic -> front-end display

   Technologies
   -------------
   enterprise level technologies
     Java
     Spring MVC
   UI component
   interaction with API

   Logistics
   ----------
   mentorship from former cs130 students
   direct experience with enterprise level engineering practices
   setup guidelines
   onsite development workshops
   Git/Github
   various methods of communication between teams and Connexity


3. International Business Machines (IBM)

   Project Lead: Gergana Markova   gmarkova@us.ibm.com
   Dean Okamura    dean.cs130@gmail.com
   Michael Stein   mastein@us.ibm.com
   Neil Sahota     nsahota@us.ibm.com

   team project planning
   collaboration, networking
   rapid decision making
   open source community involvement
   agile development
   globalization awareness
   design thinking and user experience
   reserach and resources evaluation

   Cool cs130 IBM projects
   ------------------------
   Barcode Wallet
     -> Apple Pay
   Volunteer snack delivery service
     -> Uber
   Jazz festival
   Tutoring website enhancements


4. Anderson School at UCLA
   
   cyber security for small businesses
   

=============
Apr. 1, 2016
=============

Jae Lee
TA OH: 2-4 PM Wed.

name
skills/resume
top 5 project preferences


requirements gathering exercies
--------------------------------
Paul Eggert, senior FAA admin
hire company with software developers to solve problem with software
probably some hardware

problem: drones
make country air space safe, aircraft traffic
worst: plane crash, loss of life
dealing with drones because they are threat to air traffic safety
report of drones within few meters of commercial aircraft
afraid of crash and people get killed

software solution to part of this problem
manage drone issues
in realitiy drones can be small to much larger unmanned aircraft
want system to keep track of all of this stuff
not too worried about large drones, who have lots of money
most worried about small drones

require drones operators to register their drones on website
pay $5 for registration
not every drone use has registered
fine of operating drone without licence

should be able to grab drone serial number and find owner

have website with serial number with names and everything else
really unorganized about this
data sitting in server, which is not very useful

senario: firefighters putting out fires in mountains, but spotted drones near
facilities, so couldn't fly
want to find the owner of the drone

want software that gives information to law enforcement facilities when
dealing with illegal drones

information have to be private
general public cannot find owner through serial number

mostly concerned with airport safety but firefighting case is also important
fire department can get FAA to establish a temporary no-fly zone

make data available to LAPD if needed
real goal to prevent drones from causing safety problems

may want total control over drones in the future
e.g. button that can safely land the drones and accessible to police

should have audit trail for all accesses to data and entering of data

should have images that can be matched to low quality images taken by pilots
could try to identify the signal controlling the drone

                  Wi-Fi
cell phone   --------------->   controller  --------------->  drone
                encrypted                      encrypted

Wi-Fi (WEP) is easily crackable
security on drones and controllers is terrible

drones cannot be identified by radar
want to add transponder so that if we point radar at them, we get signal back
want transponder in database for indentification
drone makers reluctant to add transponder due to weight and size
users will take transponder down
if this happens, will like to track down owner and quesiton why turned it off
cannot take down the drones, which are hard

will also work with custom-made drones
such as, must install transponder

priority: people goofing off flying into airports just to fool around

should not lose data due to disk crashes

new owner should register if bought from previous owner

how to get serial number if we do not take drone down
transponders come with serial numbers

2-3 million dollars
time frame 18 months

want this to work on police laptops and cellphones


=============
Apr. 4, 2016
=============

5. Calthorp School (Principal)
   
   standardized testingw as a big deal
   stayed in good standing

   from public school system to independent schools (private schools)

   have to ability to choose
   resources & autonomy

   schools have collected huge amounts of data
   but schools are terrible at analyzing the data
   find patterns

   data -> analysis -> trends -> areas of exploration -> questions
   -> hypotheses -> confimation/invalidation


6. Department of Psychology

   web-based tool to find career he/she is passionate about

   research team of undergrad students at UCLA
   career websites focus on interest... but what about passion?

   Pandora-like algorithm
   like or dislike

   world of work to classify


7. Beehive

   make social media analytics available to all


8. Airbnb for restaurants


9. Connoisseur

   ineffeciencies in finding next dining experience


10. Sugar-Watch
     
    glucose monitor
    how it affects the body
    partner with 2 doctors at UCLA Medical Center & Anderson School

    600 billion dollar market

    doctor & patient needs
    real time data on glucose levels


11. Aerospace Coporation

    federally funded research centers
    works in the national interest


=============
Apr. 6, 2016
=============

Why do we need requirements?
-----------------------------
* developers != users
* legal, contract reasons
* reliability/safety is crucial
* security (tricy in practice)

How much work to put into requirements?
How long/big is the requirements document?
-------------------------------------------
it dependes on the project
the more of the above required, the longer it will get


* one of the most common requirements document problem is that customers would
not want to read the long and obvious document because they are specifying
what the customers already know.


Requirement Engineering
------------------------
applying sound engineering principles to come up with requirements documents

                     requirements
    stakeholders <---------+--------> design
                           |
    - everybody who        |
    cares about            |
    the software           |
    - users/managers       |
                           |
    system model <---------+

    - in developers' heads


Good Requirements
------------------

- are testable (once system is implemented)
  ideally would want to make it quantifiable
  realistically turn it from something vague to something less vague
  "build a system where the UI is userfriendly"
    this is not testable
    change to something testable
    we could test them on users

- are feasible (in the indented environment)
  make sure requirements are actually doable
  do not make NP-complete requirements

- don't conflict with each other
  conflict are not obvious
  come from different parts of the requirements document
  gathered from different parts of the customer organization
  inconsistent requirements documents arise with conflicting user intentions
  conflicts may not be obvious and may reflect conflicts among users

- are attributed (to specific source)
  can go to any requirement in document and see who is responsible
  each requirement should be attributed to specific source
  should know who to ask if there is a problem in the requirements

- are bounded
  do not want to have requirement that is infinitely hard to satisfy
  should know when the requirements are satisfied
  "as fast as possible"
  do not want software engineers to develop "forever"

- are unambiguous
  ambiguous: so few requirements or so poorly stated so that they can be
             interpreted in many different ways
  should avoid the ambiguity (English is ambiguous)
  need somthing functional

- are essential
  Aristotle: get at heart/core of the story
             find out what really matters
             this is called the "essence"
  good requirements should focus on the essential part of the application
  and not trivial details

- are specified at the user level
  write in natural language that the user understands
  do not write in code-like languages such as Java, C, Shell Script

- match the system's vision
  when you try to change the world with your application, you should know what
  the world looks like after your application

- are prioritized
  an elaboration of "are essential"
  some requirements are more important than others
  in practice, some requirements conflict, but priority specifies direction

- are validated
  requirements are checked
  feasible: done a feasibility analysis
  unambiguous: gone through whole document and checked for ambiguity
  validated: we have to validate the validation process


Types of Requirements
----------------------

(A) funcitonal
    what the software does
    behavior of software
    get support from customers about funcitonal requirements

(B) nonfunctional
    other constraints on system that are less obvious because they do not
    initially seem to have anything to do with what software does
      security
      reliability
      performance

(A) user
    imposed on system by end-users of application
    will be able to look at user applicaiton and verify

(B) system
    more detailed
    more peopled are affected here
    audience is people that want to make sure system work
      developer
      operation staff
      finance
      managers


Phases of Requirements Development
-----------------------------------

(1) inception

    some things may sound obvious but are easily done incorrectly or not done
    
    (a) identify stakeholders & their viewpoints
        stakeholders may not want to talk to you
          e.g. prisoners in prison
        have to indentify everyone who have something to do with the project
    
    (b) find agreements & disagreements
        get a good feeling on everybody who are using the system
        may have completely different opinions between departments

    (c) break the ice by asking "dumb questions"
        context free questions
        indicate that you don't understand the field
        need humility
        ask "dumb questions":
          about goals and benefits (need to know why)
            "how are you going to make money with this?"
          about the problem
          about communication activity itself
            "did I ask all questions?"
            "are there any questions I have left out?"

(2) discovery/elicitation

    (a) use a well-defined procedure
        - have meetings with agendas & prepare for the meetings
          specialized training -> requirement facilitators (bridge gaps)
        - define problems, pieces of solutions, in user-oriented way
        - write everything down
        - iterate -> multiple meetings
          come up with draft document

    (b) produce
        - scope of requirement
          specify boundary of what to do and what not to do
        - feasibility analysis
          show that requirement is feasible in document
        - justificaiton of need
          why the requirement is needed
        - stakeholder list
          characterization of stakeholders & their viewpoints
        - environment characteristics
          what the system will operate in
          where the system will be running
        - use cases
          little scenarios of where the system will be used
        - constraints
          any sort of extra high-level constraints that are not obvious
        - prototypes
          actually write some code as part of requirements discovery
          may build end-to-end prototypes
          tend to justify feasibility
          "initial testing"

    (c) software requirements document (IEEE standard for requirements)
        contains
          - glossary
            standard nomenclature for problem
          - user requirements
            "normal" requirements
          - high-level system architecture
            document understanding of system model
          - system requirements
            stated in terms of high-level system architecture
          - system models
          - system evolution
            potential changes to the requirements

(3) negotiation

    come up with too many requirements and can't satisfy
    ideally they are prioritized but practically not that easy
    so we have to negotiate with clients

    - want win-win situation
    - key role of written requirements

(4) validation

    list of things we want out of requirements
    check consistency, completeness, etc.

    via. reviews
         prototypes (little programs to test)
         test cases


==============
Apr. 11, 2016
==============

test driven development
buggy spec, if we explore all possible test cases, we can fill in the spec
test debug the spec before writing the code
it is simpler to write tests than writing code
now we can find the bugs in our spec faster

do you use test driven development?
we don't always practice what we preach. - Paul Eggert, Ph.D

if we have real-time constraints, this needs plan driven development
safety systems also requires plan driven development

the development team gets bigger and more parts of the software are not under
your control, and you can't continuously integrate.


goals of software engineering
------------------------------
(1) understand your problem
    much of the software engineering activity is devoted to finding the
    problem that we are trying to solve, which is easiest to get wrong

(2) design is crucial
    design better be there when we are done
    we should know how the software was designed

(3) quality
    software should always be high-quality

(4) maintainability
    software has to be something that we can fix, improve, refactor

(5) work across a lot of domains
    shouldn't be good for just one thing (web, realtime, embedded, system apps)


software engineering principles (Hooker)
-----------------------------------------
(1) provide value to users
    not always obvious

(2) keep is simple stupid (KISS)
    when in doubt, use the simpler approach
    keep code as simple as possible

(3) have an architectural vision
    don't just look at little picture

(4) plan to get hit by a bus
    do not assume that your software project will have you on it
    other peoples may take over it
    somebody else may be maintaining your project
    if it's important, always write it down
    be ready to be replaced

(5) be ready to change
    designing and building software should not be like building the pyramid
    it should be able to mutated

(6) plan for reuse
    when you build your code, assume that it will be successful and you
    or other people will reuse the current code
    write code that can be reused in other systems

(7) THINK before doing
    don't just code because it feels good to type keystrokes
    think before you build the code, before it's too late

Sommerville likes 1,4,6, and

(8) worry about dependability and performance
    obvious yet important
    when you worry about dependability and performance, you are bringing to
    the table software engineering strength


Developers             Managers
------------------------------------------------------
I wanna code           ensure it does what user wants
McConnell              Sommerville

programming
textbooks
S.E. theory


software construction
----------------------

        problem definition                 corrective maintenance
        requirements definition
      --------------------------------------------------------------
                                  detailed design

      construction planning          coding          integration

                  unit testing     integration testing
      --------------------------------------------------------------
              software architecture            system testing

(1) get your prerequisites right


Plan to throw it away; you will anyhow - F. Brooks
McConnel disagrees and thinks that we should make code work
software is not authored in the usual way
software is edited (like an encyclopedia)
don't plan to throw the whole thing away


collarborative development
---------------------------
(1) focus on cost-effective defect-detection
    bugs will be the normal way of life
    will spending more time fixing defects than writing code
    reduce # of defects as many as possible

(2) collarborative practices do best on defects resistant to traditional tests
    can you break up the project


pair programming
-----------------
pay 2 people's salaries to write one program
  one programmer K has the keyboard and the house
  one programmer J just kibitzes
    find errors quickly, early when they are cheap to fix
    if we wait to review, the cost goes up
    reduces defect removal tests
    have immediate feedback
    back-and-forth is fast
    requires 2 bus hits

guidelines

(1) match pairs
    makes sure the two people are comfortable around each other

(2) rotate
    switch roles

(3) Keyboard = tactics
    Kibitzer = strategy

(4) don't let the kibitzer relax
    make pair programming sessions short

(5) don't use it for everything
    not all things are suited for pair programming


formal inspection
------------------
gold standard for software review (IBM)

(1) focus on defect detection not correction
    formal inspections are expensive
    hard part is mostly finding bugs
    so focus on hard part

(2) use a checklist to focus reviewers' attention
    use different checklists and measure how well each checklist performs
    checklist will depend on problem domain

(3) reviewers prepare for meetings
    have multiple reviewers for reviewing system
    give code ahead of time to read independently and come up with questions
    in the meeting, combine the reviews, big merge of question list

(4) all participants have roles
    moderator: requires most training, competent to organize reviewers
    scribe: keep track of what is set
    reviewers (2-5): reviews the code
    author: usually doesn't participate, but also nice to have there
    managers are not participants, inefficient

(5) time and efficiency
    100-500 lines/hour
    < 2 hours/meeting
    code reviews are very expensive


code walk-through
------------------
code readign
demos (dog & pony)
"demo or die"


software process
-----------------
the set of heuristics
developers' heads (extreme approach)

(lisp (code))
lisp code in which developers are subroutines
this doesn't work

framework for what goes on in developers' heads
dynamic perspective - phases, in some sequence
  communications (requirements)/planning/modeling/construction/deployment
practice perspective (umbrella activities)
  - quality assurance
  - reviews
  - configuration management (how system/requirements are configured)
  - project tracking (keep track of what's done/not done)


plan driven vs. agile approach


==============
Apr. 13, 2016
==============

different engieers have:
different terminology
different world views on how things work
different terforce
  software engineers want to work software engineers get more jobs

trading systems
nobody is in charge
if one system decides to change, others will have to deal with it

conceptual desing --> procurement --> development


software processes
-------------------

(1) plan driven
    
    (a) waterfall model

        | requirements |
        +--------------+
                
          | architecture |
          |    design    |
          +--------------+

               | component |
               |  design   |
               +-----------+

                     | coding |
                     +--------+

                      | unit testing |
                      +--------------+

                      | integration testing |
                      +---------------------+

                                | system testing |
                                +----------------+

                                  | acceptance testing |
                                  +--------------------+

        finish each part before doing the next part

        have a strict client who knows what they want
        then the waterfall model is suitable
        the requirements won't change
        it is inflexible
        going to spend a lot of time twiddling thumbs and waiting

    (b) incremental waterfall model

        repeated, parallel waterfalls
        before first version completes we start gathering requirements on next
        version to minimize the time to release version 3

    (c) spiral model

        increments "spirally"
        goes in organic way

            
              +---> plan -----> model ---------+
              |                                |
    +-> communication                   code <-+
    |                                     |
    +-----------> deliver <---- test <----+


        intent: each level of spiral is a major rethink such that the previous
                versions doesn't matter too much
                do risk analysis before next spiral
                can revert if necessary

    (d) concurrent development

        break task into subtasks and resolve dependencies to concurrently
        develop on the subtask with multiple waterfalls and merges


(2) agile developemnt
    
    rebellion against plans

    promote: adaptability
             self-organization (team will organize itself to get work done)
             collaboration & communication
             working software (new features) every two weeks

    over: software processes
          software tools
          documentation
          planning

    developer teams need to select the work quantity
                        

XP (eXtreme Programming) framework activities/phases
-----------------------------------------------------
                              +---+ <- customer sets the value <-+
planning => set of stories -> |   |                              |
                              +---+ <- dev team sets cost        |
                                       if > 3 weeks, split       |
                                                                 |
                                                                 |
                                           can do high-value first
                                            can do high-risk first
                                   can make commitment to a series

design => set of class designs -> CRC cards + spike solutions (running code)
                                     |
                       (class responsibility collaborator)

            spike solutions to test out a risky part of solution early

coding => code
          unit tests
          pair programming
          refactoring
          tinderboxes

testing => purely to acceptance testing


XP values
----------
communication - informal, verbal
simplicity - don't over-engineer, code for today
discipline - "courage" to say no to client to keep things simple for goals
feedback - listen to clients, fellow developers, and software
respect - respect for clients, fellow developers, and software

downside of XP (from a plan driven guy)
----------------------------------------
lack of a formal design
lack of a formal requirements
volatile requirements (cause a lot of unnecessary work)

downside of XP (from insider)
------------------------------
1. inertia - fighting
2. refactoring /simplfying
3. prioritization is hard (especially in large organizations)
4. some developers aren't good collaborators
5. some clients aren't good collaborators
6. tinder boxes can't catch everything


system engineering
-------------------
software engineering + everything else

                        hardware
                        networking
                        databases
                        documentation
                        people
                        procedures

* systems nest

figure out how to meld together components and build systems out of systems
have multiple levels of nesting (at least 4 levels, usually 5 levels)

[WORLD VIEW]
  enterprise strategy
  overall goal
  how to survive for the future twenty years

[Domain View]
  business area desing
  interested in making sure students graduate on time

[Element view]
  subsystem to help business to work
  business system design
  build tracking software and hardware to track students through degree

[detailed view]
  correspond to traditional values of software engineering
  construction and integration


system engineering phases
--------------------------
conceptual desing => system vision (feasibility + proposal)
procurement => regulations, competition, budget, buy vs build
development => e.g. waterfall
operation => system in use, bug reports, fix things while system operates
             flexibility + adaptability are key
             assume mistakes will happen & system will evolve


system modeling
----------------
Hatley-Pirbhai model


+------------------------------------+
|           user interface           |
+---------+---------------+----------+
|         |   processing  |          |
|  input  +---------------+  output  |
|         | maintainence  |          |
|         |   self-test   |          |
+---------+---------------+----------+


==============
Apr. 20, 2016
==============

$ top

  PID   CPU   CMD
  ...   ...   ...


describe a solution for the observer problem
---------------------------------------------
when a process changes, how do we notify the top program?
notification of change efficiently
want to decouble the observers from the doers


AbstractSubject             AbstractObserver
  addObserver()               notify()
  removeObserver()
  notify() {
    for o in observers:
      o.notify()
  }

ConcreteSubject             ConcreteObserver
  getState()                  State

can have serveral observers observing one subject


what are Gang of Four of design problem
----------------------------------------
name
description of problem
description of solution
consequence (pros & cons) of the pattern

communicate with fellow software developers with short names


what is the Liskov Substitution Principle
------------------------------------------
have a parent class P and child classes C1 C2 C3
should be able to substitute C2 for C1 and the program should work the same

  if (p instanceOf(C3))
    print("got a C3!");

this violates the Liskov Substitution Principle


accidents vs. essence
----------------------
core of design is essence
want to keep essence small as possible
want as few accidents as possible


call-and-return architectural style

  subroutine calls
  1 instruction pointer
  more tightly coupled, fits in well with C++, Java

message-passing architectural style

  send message to server, eventually get a respones
  N instruction pointers
  loosely coupled, fits in well with Smalltalk
  usually more advanced and fancy
  trend is going more towards this way
  * should write a programming language on this

repository architecture (DB)

  send data in to database and others can pickup whenever they want

layered architecture

  build applications in terms of layers on top of machine
  each boundary gives abstraction level

  +-----------------------+
  |   Python libraries    |
  +-----------------------+
  |  Python interpreter   |
  +-----------+-----------+
  | OS Kernel | C library |
  +-----------+-----------+
  | machine instructions  |
  +-----------------------+

client-server architecture

  may be attached to DB
  connected to network
  have a client attached to display, keyboard, mouse
  clients make requests to server and get response back
  server is in charge, responsible
  clients can only issue requests

event-processing architecture

                 events
                    +-+-+-+-+-+-+-+-+-+-+ 
  outside world  -> | | | |*|*|*|*|*|*|*|
                    +-+-+-+-+-+-+-+-+-+-+
                                      event queue
  event processing

    for (;;) {
      e = remove_queue();
      handle(e); // this part must be fast
    }            // could do some work, then signal an event!
                 // to handle large events, partition and add to event queue


distributed architecture
-------------------------
+ increased availability & reliability & fault tolerance
  high availability means system is up at most times (0.999 availability)
  failures of individual components don't cause performance degradation
  if one server crashes, system can keep running with other servers

+ scalability via concurrency
  this comes up even in a non-distribute system
  always an issue in all systems
  common impression is that distributed systems will work better with more
  servers, but this is not the case as systems do not always scale

+ openness
  acheived through protocols
  key way to glue together distributed architecture

+ resource sharing
  can support many different kinds of services on the same platform
  machines support many different users simultaneously
  this saves money & resources
  key driver behind distributed architecture

BUT MUST CONSIDER THE FOLLOWING ISSUES

- failure management
  keep track which part of system is not working

- quality of service (QoS)
  how to specify quality of service
    response time
    requests/second

- security
  safeness of system
  
- transparency
  do users know it's distributed?

- how to scale
  geogrpahy, where to put the servers?
  manageability, a million servers -> configure by hand?

- how open, really
  specialized, proprietary protocols


distributed architecture examples
----------------------------------

master-slave

                  master
  -------------------+-------------------
  +-------+ +-------+ +-------+ +-------+ 
  | slave | | slave | | slave | | slave |
  +-------+ +-------+ +-------+ +-------+ 

fat vs. thin client
 
  fat clients contain lots of state, code, processing
    less server load

  thin clients have little state (stateless), code, processing (may have cache)
    makes it easier to run on different platforms
    makes app size smaller
    security gets better since most of computation is done on server


multi-tiered-client server
  
  layered architectur applied to client-server

  browser
   |
  load balance
   |       |
  cache  cache
   |       |
  application server
       |
  database server


peer-to-peer (P2P)

  internet originally designed that there wouldn't be client or server
  "clients" only (peers)
  run mostly because peers need to interchange information with each other
  without the need for central server (control catalog) to manage them
  can have helpers (semi-centralized architecture)
  security is a problem, trust can be issues


software as a service (SaaS)

  applications on servers controlled by other organizations
  how to glue together lost s of little services
  outside your control
  rely on other services to get your service to work, what if it goes down?


management + architecture
--------------------------
- buy vs. build (any architecture)
- change management
- risk identification
- feasibility analysis
- quality requirements ("over-engineering")


construction decisions
-----------------------
- programming language
  should have at command many different different languages
  different languages are meant for different work

- programming conventions
  hard to change after initial coding
    
    $ foo --version  # should print version and exit
  
  can control this more easily than the language

- programming tools
  IDE (Eclipse)


deisgn
-------
history of software design
  modular program       1950s
  top-down development  1960s
  structured programming ("goto-less programming")
    sequencing:  s;s;
        choice:  if, case
          loop:  while

  Goto considered harmful, E.W. Dijkstra (1967)
  first mentioned by D.ValSchorre (1962)


==============
Apr. 25, 2016
==============

design
-------
design is a heuristic process, not an algorithm
heuristic: often works but sometimes doesn't, never works first time
we should always have to iterate
"wicked problem"
you don't know statement of problem until you've solved most of it

managing complexity, both essential and accidental
keep both to a minimum to get it work


Emacs bug
----------
C-x 8 RET LATIN SMALL LETTER A WITH ACUTE RET
C-x 8 RET 201C RET

but what about BED, which is "bed" and also a valid hexadecimal number
had to find the solution first before Eggert figured out the problem


important design concepts
--------------------------
(1) aspects
    if you have something of a concern about the design scattered all over code
      memory leaks, allocating memory all over the place
      cache control
      user authentication
      internationalization

(2) modularity
    redo your process or your software or both so that you can isolate these
    aspects/concerns into a small set of modules
    suppose we have 1 line of code/module, then code can;t be understandable
    should pick appropriate size for a module

(3) information hiding
    modules should not see each others' information
    ensures functional independence

(4) abstraction
    build higher-level layers with lower-level layers

(5) refinement
    = anti-abstraction

(6) patterns
    common refinement methods

(7) refactoring
    you messed up the modularity, fix it 


design patterns
----------------
high level informal relationship among
  context -> problem + solution <- design force

e.g. factory method pattern
     context: lots of related classes need to create objects
              not necessary the same classes
     problem: standard framework to let each class decide on its own what
              object to create (anytime)
    solution: static method of a class C, returns C
              but actual object may be a subclass of C
design force: creation via inheritance
              relatively simple; often evolves into fancier pattern
              perhaps evolve into protocol pattern (prototype for delegation)

              "hash-consing"
              (hash-cons a b)
              Java, C++ doesn't support this


categories of design patterns
------------------------------
(1) creational
    factory, prototype, singleton, abstract factory

(2) structural
    integrate existing into larger structures
    container, adaptor, pipes & filters

(3) behavorial
    communication & responsibility
    visitor, event listener, interpreter, model view controller (MVC)

    model: entity, "business classes", modeling what's in user's head
    view: boundary, manages how to present business to users (UI components)
    controller: glues together model + view, manages units of work


class + interface issue
------------------------
* beware erosion of abstraction
  can often occur when cost of refactoring is too high
  may be in environment where you have to talk to boss to add class

* beware if
    your class name is a verb
      has only actions
    your class has no methods (structs in C++)
      no actions
    your class does everything
      not taking classes seriously

class design suspicions
------------------------
* be supicious if you see:
    repeated code in subclasses
    base classes with just one subclass
    base classes with > 10 subclasses
    overriding methods that do less than what parent does
    more than 7 levels of inheritance
    multiple inheritance


=============
May. 4, 2016
=============

validation tests whether code matches requirements
  may be hard in practice because of communication problems
verification tests whether system works correctly

configuration management
-------------------------
baseline
  there can be many baseline versions
  may have many components, each comes in a version
  have to pick version for each component
  this is a baseline

codeline
  line of development of individual components
  make sure it is reliable so we can build future systems on it
  codelines aren't actually 'lines',
  but rather acyclic graphs because of branching

version control
----------------
not just code but also all aspects for program
  requirements
  tests
this is often broken by developers

pessimistic     clashes will be bad if two people work on same module
                involves locking to avoid race conditions
                works better with few developers & many modules

optimistic      let two people edit same module, it probably works
                if collisions happen, we do merging
                do not avoid races, but can merge changes to resolve races
                works better with many developers & few modules

centralized     one repository, must change this repository
                change does not exist until you change central repository
                database-oriented
                used by CVS ...

distributed     no central repository
                each developer has their copy of repository
                copies are not necessary in sync with each other
                used by git
                problems in this approach
                  copies are not synched, merging is needed to synchronize
                  the patch becomes your programming element
                  what about patches for patches

git log timestamps are not in time-date order
property of being distributed and de-centralized
can import patch that was dated earlier than when patch was applied

properties of version control
  isolation
  undo/revert
  update (apply patch)
  logs
  backups - are key! test them!

system building
----------------
development platform (record this in version control)
build
target

automation is key
shouldn't require many by-hand building


automating builds
------------------
sh - Lisp - chef
  series of commands to execute
  special case of 'make'
  this is easier to debug though (conventional)

make [-j8]
  dependencies
  directed acyclic graph, explore the graph in parallel
  different outputs every time we build

* project specific

autoconf
  configure.ac (100 lines) --> configure (50000 lines)

automake
  Makefile.am (100 lines) --> Makefile (10000 lines)


testing
--------
testing is unnatural!

goal - success: find bugs!

you typically cannot use tests to prove absence of errors
exhaustive testing only works in small cases

  bool not (bool b) { return !b; }
  int add (int a, int b) { return a+b; }

testing itself does not improve software
it only exposes weaknesses

tempting to under-test

we should test everything!
  code
  specs
  design
  tests [!]

testing is a strategy that you can apply to everything

test-first programming
when to test
  late
  early
  often/always

test driven development (TDD)
  + unit testing
  + converage
  + don't need debugger! (GDB) (some truth to this)
    small changes -> unworking program (easy to find bug)
  + improves documentation
    (1) the test case are the documentation!
    (2) the act of writing down tests tells you what to write in documentation

  - doesn't scale well
  - doesn't work for graphical user interfaces (GUI)
  - doesn't work well for non-deterministic programs
    TDD originally was designed for single-core machine programs
    coming from underlying system, multi-core races
    GUI can be categorized as non-deterministic

test tools
-----------
test data generators
  exist to provide test data for other programs

project-specific scaffolding
  take down after building is built
  supply tests
  build the tester to generate tests

coverage monitor
  $ gcov on SEASnet
  run you program and will show how many times each line of code got executed

platform monitors
  change platform in simple/complex way and see if it passes the same tests
  $ gcc -m32 (generate code for x86 instead of x86-64)

records + logs
  log all tests you run
  keeps a test database that keeps track of all passed/unpassed test
  along with all its parameters and return values


IEEE standard terminology (withdrawn)
--------------------------------------
error      mistake that developers make in their heads
fault      manifestation of error, mistake in the code due to error
failure    fault gets triggered in production and user sees it

error -> fault -> failure
error <- fault <- failure (debugging process)


=============
May. 9, 2016
=============

Should be only when project should be controlled

consistency
honesty
inclusion
respect

project management danger signs
--------------------------------
if you see such danger signs, it may be time for you to leave

* managers and developers avoid best practices
  you know you should do test-first development but people don't want to test
  if there is a general tendency to avoid this, then there is a problem

* sponsorship was lost or never present
  should have backing for this project (inside or outside)
  but sponsor ran out of money or person left the company

* project lacks people who have right skills
  database administrators are working on user interface redesign

* users don't want the software
  annoying software (ads, security software)

* business needs change
  needs changed completely
  idea for funding is no longer supported
  different division argue with each other
  VA Linux (huge boost on first day)

* unrealistic deadlines
  promised that software has to work by deadline

* chosen technology changes
  Android -> iOS

* changes are managed poorly
  ship out new release but forgot to inform customers about changes

* product scope isn't defined
  trying to nail down requirements but haven't figured out whether a
  particular feature belongs to this project or another
  take fuzzy area and figure out sharp line for project

* developers don't understand the customer needs
  writing code, given spec, but don't get why it's there
  think you understand customer need but actually don't

solution elements for software project management
--------------------------------------------------
(1) people

    good staff + good environment
    big enough screens in work station
    fit dev roles to personalities
      - outgoing (love people!)
        talk to customers, negotiate
        manage a project
        have the best correlation with software quality
        people skills are important in building high quality software
      - task oriented (love to program/building stuff!)
      - self oriented (what's in it for me?)
      - neuroticism (how well am I doing?)
      - openness (open to new ideas)
      - argreeableness (how like to agree with others)
        -> job satisfaction
      - conscientiousness (treat tasks seriously)
        -> job satisfaction
      - autonomy
      - meterdependency
      - conflict
    provide motivation, coordination, organization into teams
      toxic teams (weakest link dominates)
      jelled teams (people working together, strong links fill in weak links)
    
    people have different productivities
    order of magnitude difference in productivity
      5:1   ratio, even in large teams
      25:1  ratio for individuals

    Pareto principle (80-20 rule)
      have 80% contribution from 20% developers
      hire the 20% of the developers
      find the motivation the other 80% need

    Maslow motivation hierarchy
    ----------------------------
       self-realization
         esteem needs
         social needs
         safety needs

      safety needs      "do better or you're fired"
                        simple, crude, hopefully not needed
                        not that effective
                          get worried, apply funny tricks to seem to do better

      social needs      social networking, likes meetings
                        we have cities so people can meet with each other

      esteem needs      recognition by peers that you're doing good work
                        get the corner office, better pay, awards, nice parking

      self-realization  self-help, responsibility
                        realize that they can develop by themselves

(2) product

    * bound the scope
      huge part of product management
      draw line between product and non-product
    
    * decompose the problem into subproblems
      draw line between subproducts

(3) process

    organize and control by which your software is being developed
      easier than people management

    * select appropriate process model
      don't choose agile method for air-traffic control
    
    * adapt the generic model to specific environment + projects
    
    * WWWWWHH (Boehm), also W^5HH
      who  what  when  where  why  how  how much
      if these aren't answered then you don't know how process works

(4) product

    * understand the problem
      get the right team
      set realistic and clear objectives/goals

    * minimize turnovers
      let the team do the right thing
        focus on bottom-up process management
      emphasize quality
        start this from first day and not at the end

    * measure progress
      need to know what you're doing and how well you're doing it
      change management
      metrics

    * keep it simple (KISS)
      too easy to complicate things in software
      avoid risk
        simplicity to avoid risk
      focus on complicated/risky areas

    * review what went wrong/right
      don't just fire the project and forget!
      make it better next time

scaling issues
---------------
O(N^2) communication overhead
  as project scales, things working in small project starts failing

probability that a random chosen programmer in the US
will be working in a project of this size

  1-3: 5%
 4-10: 15%
11-25: 15%
26-50: 15%
  50+: 50%

productivity |  \
  kLOC/year  |   `
             |     `
             |       `
             |          `
             |              -
             |                  -
             |
             |
             +-----------------------------
            project size (kLOC) (on log scale)


risk management
----------------
~= estimating variance in project cost

  how to estimate variance?
  --------------------------
  risk table

  risky event                probability P        impact I (qualitative)
  ------------------------------------------------------------------------
  lead developer leaves      0.05
  stakeholder vanishes
  over AWS auota


  impact on qualitative scale
  1. catastrophic
  2. critical
  3. marginal
  4. negligible

  if impact is a cost then we can compute  (P*I)

system testing
---------------
requirement most planning
  harder to change than simple software testing

recovery testing
  for a fault-tolerant system or highly-available system
  make random components fail

security testing
  get penetration teams to try to break into it
    social engineering is the best way to break into system

stress/performance testing
  put it under large load to make sure it behaves correctly
  best done at entire system level
  need proper instrumentation


==============
May. 11, 2016
==============

A * SIZE^B * M
---------------

B: on a scale of [0,5]
add these together, divide by 100, then add 1.1

risk analysis?
development flexibility
preceentedness
team cohesion
process maturity


M: on scale of [1,6]

personnel capability
personnel experience
reuse required
platform difficulty
schedule
support facilities


project planning
-----------------
(1) resource allocation
    have to give resources to a project to get it work
    this is 'people' in a software project
    large company: involve drafting people to get the job done
    small company: use all you have 

(2) scheduling
    when to do what
    as it scales, it gets complex
    unrealistic schedules
      are the project manager's faults
      although tempting to come up with schedule to make customer happy
      learn to resist the unrealistic schedules
      avoid these schedules by negotiating with the customer
      cost models are what we use in negotiations
      learn to redo cost model as the project goes on

(3) cost estimation
    resource allocation + scheduling = cost estimation
    in practice, project planning is rarely disciplined
    it is a interwined consideration
    order of the 3 will not be the same
    may have cost then determine resource + schedule

typical components of a development plan
-----------------------------------------
(1) team organization
    who will work on project and how they will collaborate

(2) risk analysis
    come up with idea on which parts of project are risky

(3) resource requirements
    not just software but also hardware and networking, etc

(4) work breakdown
    unless you have simple project, you have to split up the work
    among developers to do it together
    come up with tasks, milestones, deliverables

(5) schedule

(6) reporting mechanism
    usually forgotten
    crucial to keep track of what work has been done
                             what problems to worry about

hierarchical
  probably the most efficient way to organize software development
  not always needed for small projects

synchronous
  have a lot of people working in parallel
  glue it together in the end and it works
  minimize effort coordinating
  maximize effort getting work done
  has problem with scaling

agile
  starts up synchronous
  then teams self-organize into small hierarchical trees

major issues in a organization
-------------------------------
lines of communication
  among whom?
  email? IRC?
  don't overuse communication
  be efficient in communicating

selecting members
  pick good members for team
  have complementary personalities
  technical competence
  interviewing skills are essential

risk assessment
----------------
need to keep doing as project moves on

* preliminary
  done during requirements gathering
  haven't decided on implementation
  have to do this to know whether to commit to project
  most important to get right

* life-cycle
  done during system development
  know the implementation but don't know the usage fully
  can predict how the software will be used

* operational
  done during production
  software is run with real users
  goal is to resond to production issues
  quick response to issues as they arise

risk categories
----------------
instead of looking at how the risks are going to hit you
look at where risks are coming from

* business
  may be driven out of business by large companies
    Google competes!

* product
  something goes wrong inside software you've written
    garbage collector doesn't scale

* project
  have nothing to do with actual product you're building
    leading developer leaves

risks can overlap although we may think they're independent

sample resource issues
-----------------------
* buy or build?
  should you reuse software or simply build it yourself
  major issue

* inhouse or outsource?
  write it in your own team or outsource it to other teams

* different combinations of the two above
  20 projects -> 2^20 combinations
  how to prune intelligently

deining your tasks
-------------------
class-based
package-based

tasks nest
tasks have dependencies

dependency graph
we can generate a schedule with this graph

critical path analysis
scheduling time -> Gantt chart

when you're late
-----------------
* add resources to project
* add time (delay due date)
* decrease scope

how to price a software project
--------------------------------

C    costs (usually unknown)
P    profit
-----------------------------
C+P  done

we typically guess the cost or adjust the software to match our budget
adjust software to match budget (relies on trust)

general rules for project estimation
-------------------------------------
average large project is 1 year late and 100% over budget

(1) keep + use records of previous projects
(2) use several methods, and cross-check
(3) assume things will go wrong during development
(4) if possible, develop + estimate incrementally

factors affecting cost
-----------------------
(1) size
      $ wc # number of lines
    length of API
    helpful to estimate size before writing the code
    count statements instead of comments
      ;;;;;;;; is totally valid in C

(2) complexity

(3) requirements analyst capability
    requirements analysis is among the hardest parts of software analysis

(4) prgrammer capability

(5) CPU time / storage constraints
    typically bites during embedded application

(6) personnel turnover
    the larger the project, the more likely this is going to happen

(7) platform variability
    affect cost model quite a bit

(8) team experience
    application area
    language and tools
    production platform

(9) use of software tools
    do we have good static tools that will help find bugs before runtime

(10) location / communication

(11) motivation


* Putnam Norden Rayleigh (PNR) curve


==============
May. 16, 2016
==============

2. what is the relationship between statistical testing
   and operational profiling uncertainty?

statistical testing: testing randomly with undeterministic tests
                     figure out what real-world input would look like
                     and test these inputs
                     works best where we know what use cases are

                     sort program, we count out-of-order numbers and then
                     generate similar test cases to test that

operational profile: description of what inputs should look like
                     uncertain: haven't built up what input would look like
                                low quality of statistical testing
                                since we do not have the user input profile yet

3. what is the relationship between experience-based testing
   and penetration testing?

experience-based testing: come up with test cases for unit, integration tests
                          use experience to come up with testing

penetration testing: have a team who is really good in breaking into systems
                     form of experience-based testing
                     special case of experience-based testing

4. give 3 basic methods for risk reduction in safety engineering

(a) avoid the risk, make it impossible for things to "catch fire"
(b) detect and remove, catch problem and fix it
(c) tolerate, ignore the risk and keep going


ORCA: election day application
      Romney campaign
        secret, didn't want to expose to opponent (imitate or counter)
      data mining system designed to run in cloud-like system
      clients running in election worker cellphones
      focus limited resources on aspects that will do elections most good

apps on cellphones + web service + data mining
out-of-house development
reported to a different technique
rolled out at 0600 EST 2012-11-06

got Distributed Denial of Service Attack (DDoS) on release day


dependibility
--------------
can rely on the system working on arbitrary real-world conditions
  even under adverse real-world conditions
  even if you have adversaries
assume universe is subtle

* security (for later)
* safety
  likelyhood that software will not damage users either directly or indirectly
* reliability
  likelyhood that software will match expectations
* availability
  likelyhood that service will be up
* resilience
  likelyhood that software will resist and recover from damage

* maintainability
  likelyhood that software will stay working after change
* repairability
  how fixable when buggy
* user error tolerance

dependibility engineering
--------------------------
requirements
software architecture
dependible programming
software processes
  software development process/method
  can show why .999 depenidble
  can prove that software has reached dependibility goal

security
---------
assume universe is malicious
paranoia is almost required here
security is a more difficult problem

often the most important
  can trump funcitonality
bad P.R. if it fails

cost ($) vs. dependibility (0-1): curve has asymtotic growth
choosing position on curve is a hard design decision


specifying dependibility and security
--------------------------------------
often requirements are the hardest part
easy to think you're done because you filled out all forms, but you're not
over-specify depndability
techniques
  -> identify risks
  -> analyze & prioritize risks
  -> decompose the risks into their root causes
  -> reduce them

high failure rates are sometimes ok
failure rate may depend on other factors
  may need to specify different failure rates for different causes

safety terminlology
--------------------
       accident   unplanned event causing injury
         hazard   condition that might lead to an accident
         damage   measure of cost of accident
hazard severity   what's the worst that can happen?
           risk   probability of an accident

security terminlogy
--------------------
      exploit   unwanted event causing loss of privacy, etc.
vulnerability   condition that might lead to an exploit
     exposure   system is available to be attacked
       attack   attempt to exploit
       threat   vulnerability that might be attacked ('misuse case')
      control   mechanism to defend a vulnerability

reliability terminology
------------------------
MTBF   mean time between failure
MTTF   mean time to failure
MTTR   mean time to repair

       up       down   up   down         up
---------------+    +------+    +-----------------
               +----+      +----+
                    MTBF
               |<--------->|
               |<-->|<---->|
                MTTR  MTTF

AVAIL = MTTF/MTBF >= 0.999 (3 9s)
POFOD = probability of failure on demand
ROCOF = rate of occurence of failure = 1/MTTF

safety requirements
--------------------
   primary (e.g. brakes?)
vs.
   secondary (e.g. both brake lights?)

analysis of risks from requirements
------------------------------------
fault-tree analysis
  try to categorize all bad things that can happen
  tree where leaves represent accident or hazard
              nodes represent events that leads to accident
  tree node will be AND/OR
  organize the tree to capture dependencies of the accidents
  if the analysis works well, you can assign probablities to branches of tree
  not always clear (correlation between events)
  can calculate prbability for hazard analysis table
  this makes hazards/accidents easy to explain

dependability & architecture
-----------------------------
security kernel + protection systems
  minimalize amount of software that have to be secure and put in kernel
  architecture that makes it easier to build dependable systems

self-monitoring architectures
  log the stuff that try to cross the walls
  e.g. Firefox Health

N-version programming
  create N versions for the same program
  vote for the best and the majority rules
    paying N times (or more) as much for the same program
    have discrepancies between engineers in different versions


==============
May. 18, 2016
==============

legacy systems have both code and people
code is mysterious to programmers
replacing legacy system doesn't mean duplicating what code does
but also have to figure out what people do

bad smells for refactoring
  data clumping
  duplicate code
  speculartive generality


security & dependability
-------------------------
hard to retrofit security
security risk assessment (how worried about security)
  identify assests (things to protect)
    financial, phyiscal
    prioritize assets (give value to asset)
    identify exposures
  identify threats
    attacks
    estimate cost of each attack
  specify controls + feasibility

security design guidelines
---------------------------
have an explicit security policy, and use it
  providing scope for security, which is a problem in software development
  there's always more you can do with security
  you should know where to stop

defense in depth
  used in football, military
  don't rely on one technique to defend system
  have multiple levels of defense so that attackers have a hard time

fail securely
  robust, dependable system
  deal with component failure
  a good chunk of code deals with failures, exception handling
  fail in such a way that you still have security policy
  exception handlers are big problems in security

balance security and usability
  a brick is very secure
  need software that actually works
  security is not end goal that dominates everything else
  people are willing to give up some security for features

log actions
  log actions coming from the outside world (users)
  this is brute force debugging applied to security
    `printf' statements sent to logs
  doing this under assumption that program will fail, mess up

reduce risk via redundancy and diveristy
  have multiple identical copies of the password server
  does not necessary reduce risk

specify imput format
  people don't do this all the time
  fairly mundane but is often forgotten

compartmentalize assets
  related to defense in depth but different
    put several layers around valuable assets
  break up assets into pieces
    attacker will not get all information with breaking in at once
    have more database systems, although harder to manage

design for deployment
  when a new version of program comes out, want this program secure
  no configuration, no simple default root password (`admin')

design for recovery
  when system gets breached, need a way to recover into secure system
  without posting root password `on the wall'

how to check this stuff?
-------------------------
how to make sure that system is secure?
  tiger teams, test cases

procedures for developers
  don't think security as something built into product
  security is also a property of development process
  what to do if a developer goes bad?
    log developer actions
    reuse security guidelines above and apply to development process

dependable software processes
  you not only build dependable software but also using dependable processes
  convince customer / developer / regulator that software is dependable
  
  can involve the following
    test plan + management
      need to document these
    static analysis
      describe tools and run the analysis, then list the results
    formal inspections
      formal code reviews, etc.
    formal specifications
      write down specifications for code in formal language, based in logic
    system modeling
      provide model / simulation of system in report
    review requirements + management
      management => version control
      go back and look how requirements evolved with time

    goal of the process
      auditable
      redundant
        provide multiple ways on making sure system is dependable & secure
      robust
        assume development steps will not be done correctly
        want system to recover well from failure

dependable programming
  information hiding
    private class members not to make it hard on users
    provide interior security walls
    keep part of implementation private so not visible to other parts of code
    can isolate walls into the code, fewer avenues of attack into system
  
  input validity checking
    specify format
    constantly checking validity of input data

  exception handling
    commonly used for dependability
    resume execution after some part of computation fails
    promotes redundancy

  omit tricky stuff
    floating point
      rounding errors
      computing with values that are close to what you want but not exactly
      descrepancy can lead to unreliability
    goto's
      generally a bad idea to use them
    pointers
      null pointers
      dangling pointers (freed memory)
      imposter pointers (point to memory not supposed to point at)
    subscripts
      arrays and indexing into an array
      can have dangling index in array
      all problems with pointers can exist with subscripts
    new
      creating object is trouble, similar to pointers
    recursion
      dangerous
      understood by only 25% of programmers
      structure hard to understand
    parallelism (threads)
      source of enormous class of bugs
      race conditions / deadlock
    signals / traps / interrupts
      asynchronous function calls
      can break a lot of things with signals
    aliasing
      can get aliasing without pointers
      it is helplessly confusing
      recipe for bugs
    inheritance
      breaks incapsulation
      don't have walls between parent and children
      another recipe for bugs
    polymorphism
      also very comfusing

debugging (M.W. Wilkes)
------------------------
can take half the development time
20:1 ratio of performance between experienced & inexperienced developers
connotation of dread, get little reward in publishing papers on debugging
this is the least understood part of software development/construction

how not to debug (classic mistakes)

  (1) assume the error is somewhere else in the system
      `printf is broken!'
      printf("%d\n", i) // prints out garbage where i = 2
      actually defined: long i;

  (2) fix the problem without completely understanding it
      you're in a hurry
      you despair in ever understanding the problem

how to debug

  (1) make the bug reproducable
      make the bug happen whenever you want it to happen
      it may be difficult, especially as we gain parallelism
      and undeterministic programs
      bug doesn't always happen in the program
      get rid of the nondeterministic
        $ gcc -fdeterministic foo.c  # doesn't exist
        [!] except that there is no `deterministic' flag
      
      how to make deterministic
        turn off multi-threading/multi-processing
          $ make -j  # builds in parallel, remove it
        specify the seed for the random generator
          can be really helpful to run in controlled environment
        clear all variables before startup
        reset the clock
          use a clock permanently set to 1970
          or just set the time

  (2) locate the fault
      doesn't have an algorithm
      use the scientific method
        form a hypothesis about the fault
        test the hypothesis
      painstaking process

  (3) fix the defect
      typically a relatively easy step compared to the above

  (4) test the fix
      common for debugging to involve adding new test cases
      this prevents the bug from going undetected in the future

  (5) look for similar errors
      bugs travel in clumps
      there's probably more than one floating around

useful heuristics
------------------
(1) look at code that is changed recently
    $ git bisect

      |-----------------------|-----------------------|
      ok                      ok                      ng

    do binary search for the defect

(2) prefer O(logN) techniques to search for defects

(3) budget your time
    try another technique
    rewrite the program! (only if have decent budget)

(4) one fix at a time
    general advice in software development
    can test small pieces
    apply multiple patches where each patch fixes one bug at a time

(5) keep out of ruts
    be flexible, jump around a lot
    try looking at things a different way
    `part-time debugger', come back to bug with a fresh mind
      typically more efficient

software evolution
-------------------
deliver the software (people are using)
apply a change (delta) to the software
  maybe your business changed
  maybe your technology changed
  bugs

planned vs. agile
  all plans for development applies to evolution as well
  new phase: understanding existing software
    evolving pre-engineered old code
    can be significant part of your work
    gain maximum amount of understanding for minimum amount of work

how to efficienty grok existing software
  look at its documentation
    not always available
  source code translator
    translate to other language and run the translated code (C -> Python)
    this may not work well


==============
May. 23, 2016
==============

How is Z Schema defined?

what are state variables?
what are their constraints?
what were the state transitions?

Two types of code beautifiers?

(1) changes the code, into a certain type of style
(2) doesn't change the code, gives highlighting


why haven't formal methods taken over?
---------------------------------------
first proposed in 1960s and lots of work was done in this

(1) scalability
    tends to work best when spec is small
    has problems scaling up to large systems
    large systems can't use them & small systems don't need them

(2) limited scope (e.g. no UIs)

(3) expectations for reliability have relaxed
    don't care if software is reliable anymore
    in old times, software had to be reliable to process important things

(4) there are other cheaper ways of improving reliability
    these other ways are cheaper than formal methods
      formal inspections
      hardware support (more checking, bad pointers, subscript)
      object oriented information hiding

formal methods downsides
-------------------------
(1) they're hard (need an expert)
    complicated

(2) notation is crucial
    coming up with good notation is more important than programming

(3) easy to overformalize
    once you've captured the idea to formalize
    it becomes tempting to formalize out spec completely
    cost goes up and may cost more than it's worth
    try to formalize the important part
    retain the ability to leave other parts informal to keep costs down

(4) not a substitute for competence
    will not make you competent in the first place

(5) they still need comments
    will still require informal comments

(6) don't guarantee code to be bug-free
    formally verify code satisfies the spec, you know your code matches spec
    but this doesn't guarantee that you code is bugfree, you spec may have bugs

(7) they need same software engineering effort as everything else
    put formal specs under version control
    have to review specs so to make sure they are the right specs

And yet -
  Coverity (practical)
    uses formal methods
  proof-carrying code
    gives answers + proof that answer is correct
    checker can detect malicious behavior, check proof, proof is wrong

informal specifications - problems
-----------------------------------
(1) get contradictions
    result from several meetings
    write down multiple specs
    discover that specs are contradictory

(2) ambiguities, vagueness, incompleteness
    forget to include some aspects
    all the same thing in formal specs
      nailing down set of possible worlds for implementation
        contradiction: empty
        ambiguity: too large
    spend most of effort in finding incompleteness

standard formal methods ideas
------------------------------
(1) invariant
    bool expression involving all state variables
    should always give true

(2) precondition
    bool expression involving all state variables
    should be true just before a particular state transition

(3) postcondition
    bool expression involving all state variables
    should be true just after a particular state transition

specification languages
------------------------
standard programming languages
tends not to work very well
  don't have to be executed
  they can be much higher level
    sets, for all, exists
  they are often tailored for specific domains
  they encourage  you to nail down all spec details


formal methods details
-----------------------
  algebraic
    typically for functional code
    no state
    just recursive data structures

vs

  model based
    typically for imperative code
    state machine via state variables
    invariants that have to be always true
    state transitions
    initial state predicate

    invariant, trans
      ==> invariant

    initial state, trans1, trans2, ... , transN
      ==> invariant

examples
---------
signatures
  List(Elem)
  sort List
constructors
  Create -> List
  Cons(Elem, List) -> List
inspectors
  Head(List) -> Elem
  Tail(List) -> List
axioms
  Head(Cons(V,L)) = V
  Tail(Cons(V,L)) = L

behavior driven development
----------------------------
poor person approximation to 'Z'

write down a scenario in a text file
  given: <precondition>
         bank balance is positive
   when: <state name + arguments>
         withdraw N dollars
   then: <postcondition>
         new bank balance is N less than old one

domain specific language
read text file and generate some python code

/balance is
/{check(balance > N);}

construction tools
-------------------
(1) editing source code
    Eggert uses Emacs
    others like Vim, nano, Eclipse, VS, Xcode

(2) refactoring source code
    add new argument to function
    should be easy to change function calls to it

(3) version control
     
    diff: compare two files
          diff X Y => XdeltaY
    
    patch: apply diff and get new file
           patch XdeltaY X => Y
    
    merge: merge two different files
           
           diff X Y => XdeltaY        diff X Z => XdeltaZ
           patch XdeltaY Z => M       patch XdeltaZ Y => M'

           is M = M' ?
           not always, there might be incompatible merge conflicts


    $ (emacs) M-x vc-pull  ->  returned gibberish
    
                            +----+----+
    source code in UTF-16   |    |    |
                            +----+----+
                              16 bits
    
    git diff  ->  "Binary files A and B diff"  (in ASCII)
    emacs converted this into UTF-16, and returned gibberish

(4) code beautifiers
    indent (Linux)
      can lead to many problems
      fight against other changes to source code, since it changes globally
    javadoc (Java)
      turns comments into beautiful documentation
    xref
      cross-referencing tool
      where a name is used and defined
      does auto-completion when you start typing identifier
    diagram generators
      traditionally, simple tools to generate flow chart for code flow
      state machine diagram
    lint
      cc but no code compiling
      instead style/correctness checking


==============
May. 25, 2016
==============

tools for debugging
--------------------
(1) google search
    very good choice

(2) print statements
    logging out actions
    add stuff to code to output useful information through execution
    highly recommended, even in production code, usually turned off
    want log analysis tools for this (grep)
      Apache log format
      well documented
      with ecosystem of tools to analyze

(3) memory analysis tools
    valgrind, etc.
    suspicious accesses, memory leaks
    run different version (valgrind version) of the program
    sometimes static analysis

(4) debuggers
    GDB, step through program
    avoid these if possible

    breakpoints
      jump ahead and stop at certain points
    watchpoints
      wait for program to hit a piece of data (read/write)

(5) ask a debugging expert    
    some are really good at this

(6) profilers
    gprof program
    performance instrumenting mode
    generates extra code to increment counter each time code is executed
    gprof looks at execution output and analyzes
    $ gcc -pg

(7) project-specific tools
    know that it's going to work for your particular application
      Ken Thompson: chess machine
      relied on scanned games, got many scanned errors
      wrote special debugging tool
      looked at raw data, returned possibilities with probabilties
      then fed possibilities into validator to check if move was possible
    build vs. buy
      buy from someone else of build it yourself
      sometimes it will be easier to just build your debugging tool

user interface design
----------------------
special case of design
all points on how to do design all apply to user interface design

how is it different from general software design

  crucial to learn the user characteristics before doing the design
    learn to think like the users

  some general principles for UI focus neeed

    (1) reduce user memory load
        assume users will forget, because they do not focus on application

        * have consistent user interface
          don't have differnent ways for deleting things!
          usually different designers & developers make different things

        * be compatible with existing practice
          comply to the 'two-finger pinch'
          can collide with above
            either be consistent with system or consistent with world

        * use real-world metaphors
          use from real world so they have less to learn

        * disclose information progressively
          don't have to give user at first glance
          if they may not want the information provided
            file browser
              list names
              hover mouse over it and show more
              open it to show even more
          avoid information overload

        * make context explicit
          if the user has interacted with system
          they should be able to see, just by looking at the page
          the context of the page, such as 'how I got here'
          show 'breadcrumbs', path to how they got here

        * defaults
          supply good defaults for things that users may not change
          choosing defaults becomes delicate and important exercise
          sometimes increase memory load

        * shortcuts
          can be user-programmable
          different users have different habits
          so they can have their own shortcuts for doing things
          sometimes increase memory load

    (2) have to put the user in control
        example where user is not in control
          +----------------------+
          | fatal error: exit?   |
          |               +----+ |
          |               | ok | |
          |               +----+ |
          +----------------------+

        * no dead ends
          user interface should not hang

        * flexible
          lots of options

        * undo/redo
          should have undo/redo key
          can try things out and go back to previous state
          use redo to avoid chain of undo
            A1  A2  A3 <-> A4
            A1 <- A2 <- A3 <- A4

common user interface design issues
------------------------------------
(1) response time
    should not have LONG response time

(2) help
    how can user discover how to use application as they use it
    usually added at last minute, and does not work well

(3) error handling
    when user makes a mistake/system has a problem, how to let user know
    what will the user see?
    decide which problem are worth notifying the user about

(4) shortcuts/commands vs. menu/mouse
    commonly known as CLI vs. GUI
    prompt the user or wait for commands?

(5) internationalization (i18n)
    error message text
    horizontally & vertically for different languages
    make user interface easily portable to such languages

(6) accessibility
    make application useful who have disabilities
    might have a blind user

    http://w3.org/WAI

    * provide text alternatives for any useful non-text content
      picture on page is how user will select where to go to
      <img href="foo.jpg" alt="agree to arbitration?">
      if there is no alt text, screen reader will read 'foo.jpg'
    * synchronize multimedia alternatives
      make sure all finish at the same time
    * main image/video should be clear to people with partial disabilities
      make sure text is big enough
      what about people who are red/green color blind?
    * make all functionality doable from the keyboard
      can also talk to keyboard
    * give users enough time
      for the speech-to-text interfaces
    * don't create context known to cause seizures
      > 3 flashes/second

common usability problems
--------------------------
in intranets

(1) search
    need to find out small part of larger user interface
    can use search engine (Google/Bing)
    sometimes cannot use search engine on intranet
    can use internal search engine
    some things will not be easily searchable
    should have good organization for your system
(2) login
    authentication/authorization
    getting access to stuff you need access to

how to define usability?
-------------------------
(1) more effective
    does what you want
(2) more efficient to use
    can tell it what you want to do quicker than other UIs
(3) easier to learn and remember
    minimizing the memory load for the user interface itself, not data
(4) more satisfying to use
    hardest to nail down
    more 'user-friendly', comfortable, natural
    typically need market research to establish
(5) fewer and less important errors

how to measure usability?
--------------------------
sometimes it will be straightforward
e.g. errors in UI

questions to answer before measuring

(1) know what user does and what the implementation does
    people want to do something at high level
    decompose into smaller commands and there are different commands
(2) what is the user's context?
    why would the user want to use your application
(3) what is the user's background
    who's the audience for your application
(4) what is the user's need
    what they want to do

questions while measuring usability

(1) how much traning to use system?
    training is part of usability testing
(2) does help suffice?
    can users figure stuff out on their own
    do you have to step in to help them (this can be considered as a bug)
(3) what kind of and how many error?
    can count errors
    each time user makes mistake is a success in usability testing
(4) how do users recover from errors?


what users say is less important than what they say
measure what the DO, not what they SAY!

don't partition usability testing by function

how do you know when to stop?
you don't know, you stop when the money runs out!
no this is actually not correct
need to specify the scope

suppose catch users in Los Angeles, New Orleans, New York
not useful unless application differs in regional diversity

probably useful to test different ages