Course Objectives
ENGM 732
Stochastic Models

a Kellogg2
Click on Pic for Personal Page
Mail S. Kellogg




Markov Chains

    Given a word problem, be able to develop an appropriate markov chain model.

    Given a problem description, be able to write the one-step transition probability matrix.

    Be able to classify the states of a markov chain given the one-step transition probability matrix.

    Given an ergodic chain and the one-step transition probability matrix, be able to compute the steady state transition probabilities.

    Given an ergodic chain and the one-step transition probability matrix, be able to compute the expected first transition times.

    Given a simple cost function and the steady state transition probabilities, be able to compute the long run expected average costs.

Queueing Theory
    Given an M/M/1 with arrival rate lambda and service rate mu, be able to compute
    1. the steady state average number in the system
    2. the steady state average number in the queue
    3. the steady state average time in the system
    4. the steady state average wait time in the queue
    5. the percentage of time a server is idle.

    Given a general queue with rate transition diagram, be able to determine the steady state probabilities,the average number in the system, and the average number in the queue.

    Given a queue with finite capacity, be able to use the formulas, charts, or rate diagrams to determine
    1. the steady state average number in the queue
    2. the percentage of time a server is idle
    3. the percentage of time a customer balks

    Given a queue with finite capacity, steady probabilities, and the average number in the queue/system, be able to use the revised Little's formula to determine the average number in the system/queue.

    Given a queue with multiple servers, be able to use the formulas or charts to determine the steady state average number in the queue the percentage of time one or more servers is idle

    Given a g(N) cost function be able to determine the minimum cost between two alternatives

    Given an h(w) cost function be able to determine the minimum cost between two alternatives.

    Given a G/M/1, M/G/1, or G/G/1 queue, be able to determine performance metrics Lq and Wq.

    Given a Jackson network be able to determine queue performance metrics for a network and make recommendations.

Decision Theory

  • Given a payoff matrix, be able to determine the best decision using maximin criteria
  • minimax critieria
  • Baye's Decision rule (expectation)
  • maximum likelihood criteria
  • aspiration level
  • Hurwicz's Principle
  • Given a payoff matrix with two outcomes, be able to perform a simple sensitivity analysis on the probability associated with one of the outcomes.

Given a simple decision tree, be able to
  1. determine an optimal solution
  2. determine the value of perfect information
Dynamic Programming

    Given a simple deterministic, discrete problem, be able to develop a dynamic programming model and determine an optimal solution.

    Given a simple probabilistic, discrete problem, be able to perform one to two iterations using backward recursion.

    Given a simple deterministic, continuous problem, be able to perform one iteration using backward recursion.