| CPSC 327 | Data Structures and Algorithms | Spring 2024 |
Include the steps outlined in the 15.5-step process and not other steps.
The specifications should include a statement of the task and, separately, the output, what constitutes a legal solution, and (if applicable), the optimization goal. The output should describe only what the output is, similar to the return type of a function, and not also what makes a legal or optimal instance of that output.
Targets and tactics are not about specifics of the algorithm you are developing. They are about guiding that development. If nothing more specific is given in the problem, targets should identify the brute force algorithm and its running time. For an optimization problem, brute force is to enumerate all the possible solutions in order to find the best one. Tactics are about algorithmic strategies that may be useful or which can be ruled out based on the targets. For greedy algorithms, since the paradigm typically naturally leads to a polynomial-time solution for an exponential-time brute force problem, greedy itself is the most significant tactic.
The role of the Approaches step is to determine possible frameworks for the algorithm. Greedy algorithms solve problems formulated as a series of (the same kind of) choices, which means there should be a single loop iterating through that series of choices and a single alternative picked in each iteration. The purpose of the approaches step is to identify what the series of choices is. (Not how to make each choice.)
The specific flavor needs to match the actual output of the problem. "Find a subset" means that the output is a collection of elements which is a subset of the input. The captioning problem can be thought of as finding subsets — which set of programs each employee captions — but the output isn't a subset, it is an assignment of employees to programs. This is a labelling task — each program is assigned (labelled with) an employee, and the output consists of all of the input items (with some additional info).
The greedy strategies is about the criteria for how each choice identified in Approaches is made. The goal is to identify plausible criteria — consider the input. For programs, you have start and end times, so start time, end time, and length (end-start) are all possible things to base a greedy strategy on.
Counterexamples rule out plausible greedy strategies that aren't going to work. Dismissing this step with "there isn't a counterexample because the greedy strategy is plausible" defeats the purpose. Think about what a counterexample would need to look like, and explain why that can't happen. That falls short of proving that there is no counterexample, but that's OK here — the goal is to find counterexamples where they exist and provide confidence that the greedy strategy chosen can be proven to be correct.
The main steps should make the loop structure and the series of choices clear, and the specific criteria for making each choice should now be included. Avoid nested loops, and focus on the ideas rather than implementation — the loop should be looping through the series of choices, and (only) the criteria for the choice should be stated e.g. "shortest thing next". Carrying out the greedy choice — actually finding the shortest thing — might involve another loop, but that's in the implementation and does not impact the correctness of the algorithm. There may be correct algorithms which do involve a nested loop structure, but it can be harder to work with these and there should a series-of-choices formulation with a single loop.
The Greedy Strategies step should outline all of the plausible criteria for making the choice. There should not suddenly be a sorting step introduced in the main steps (or, more properly, in the setup). Sorting defines a strategy for picking the next thing being iterated through e.g. if you are sorting the programs in some way before going through each program to assign it to an employee, the series of choices really also includes picking the program and more of a produce output approach: repeatedly produce the next output is
repeatedly
pick the next program and assign an employee to it
Now the greedy choice is how that next program is picked. There is still a choice in terms of which employee, but that seems more dictated — an existing employee is assigned if possible. (If there are several available employees, does it matter which is chosen?)
Typical special cases include when there are duplicates or ties in the greedy criteria. Consider this. Does it matter if ties are broken arbitrarily?
The Algorithm step should bring together the main steps, exit condition, setup, wrapup, and handling of special cases in one place. There should not be new elements appearing that aren't accounted for in one of those other steps. (The handling of special cases may be introduced for the first time, but the identification of those special cases should be in the Special Cases step.)
The loop invariant is what proves that the algorithm produces a legal and, in the case of optimization problems, optimal solution. For a series-of-choices algorithm structure, the invariant takes the (very) general form of: After k choices have been made, the partial solution resulting from those k choices is legal and the algorithm's partial solution is no worse than the optimal's solution. The latter part deals with optimality in the form of a staying ahead argument — the algorithm is doing at least as well as the optimal. A direct statement of "...and the algorithm's partial solution is the best" is difficult to provde because you don't outright know what the best solution is in order to compare the algorithm's solution to it. The staying ahead strategy provides a tool for the argument: if the optimal solution-so-far is somehow now better than the algorithm's solution-so-far and it wasn't before, the algorithm and the optimal must have made different choices in this last step — and that gives an opportunity to explain why there couldn't have been different choices because if the optimal's choice was available, the algorithm would have made it instead. (Or that the optimal couldn't have made that choice.)
Some examples of problematic invariants, focusing on the optimality part of the invariant. After k programs have been assigned to employees...
...the number of employees assigned is as few as possible. This is saying the algorithm's partial solution is optimal.
...every program assigned so far is part of an optimal assignment that minimizes the number of employees. Another way of saying the algorithm's partial solution is optimal.
...the gaps between each employee's show are as tight as they can be. "As tight as they can be" is a direct statement of optimality rather than staying ahead. In addition, the criteria are not well-defined (a sequence of shows will potentially have gaps between each, so what does "as tight as they can be" mean for a collection of gaps?) and are not directly connected to the goal of fewest employees. (Why tightly packed shows means fewest employees is what you are proving — if the algorithm's strategy is tightly packed shows, showing that it is a correct algorithm means showing that tightly packed shows results in the fewest employees.)
...the last program assigned to each employee is the one that finishes earliest among all programs assigned to that employee. This is trivially true or false because it depends only on the algorithm's process (what order the programs are considered in), and is not directly connected to the goal of fewest employees.
...the algorithm's solution has gotten at least as close to the end of the list of programs to assign as the optimal has. This has the right form for a staying ahead invariant, but it is trivially true — how close either solution is to the end of the list of programs to be assigned depends on the number of programs that have been assigned an employee, and after k programs have been assigned employees, that number is k.
...we haven't gone wrong yet in terms of using the minimum number of employees. As an alternative to the staying ahead pattern, "we haven't gone wrong yet" means that there is still an optimal solution consistent with the algorithm's partial solution so far, and the way to make that argument is typically to show that if "the" optimal solution differs from what the algorithm did in this step, it can be transformed into another optimal solution that matches the algorithm's solution so far. This is often trickier to argue that a staying ahead argument, so staying ahead approach is recommended if it works. (In this case, "we haven't gone wrong yet in terms of using the minimum number of employees" means "we haven't used more employees than necessary", and maintaining the invariant would require showing that if the algorithm added a new employee for program k+1 and the optimal didn't, the optimal solution already needed the number of employees the algorithm now needs.)
In establishing and maintaining the invariant, you cannot say that the algorithm's solution is optimal because the algorithm chose X — that choosing X results in the optimal answer is what proving the algorithm works is about.
For the Final Answer step, use the invariant (which tells you about the correctness of the partial solution) and the exit condition (which tells you when the partial solution is complete) to conclude that the (whole) solution is correct. If the measure employed in the staying ahead argument is the optimization criterion, this takes the (very general) form: The invariant says that after k things are done, the partial solution is legal and at least as good as the optimal in terms of the thing we are trying to optimize. The exit condition says k=n. Thus, the complete solution, which is a partial solution after n things have been done, is legal and at least as good as the optimal in terms of the thing we are trying to optimize. Since a legal solution can't be better than optimal, it must be optimal.
The Implementation step isn't about code or necessarily even specific data structures like arrays or hashtables or Maps, though it touches on that. This is the place to think about how to carry out the ideas in the algorithm — that "repeatedly picking the shortest thing next" (for example) could be implemented efficiently by sorting things by length first and then simply going through the list, rather than going through the list every time to find the shortest thing.
Well-known things don't need to be explained further, meaning that common operations in common data structures don't need need more detail. Include specifics only if important e.g. adding something to an unordered collection is O(1) and traversing a collection is O(1) per element for many types of collections, so if you only need those two operations, there's no need to specify that it is a array or a linked list or whatever — just state O(1) to add, O(n) to traverse, etc.
While it isn't specifically wrong to include too much detail in the writing of the algorithm or the discussion of implementation, too much detail obscures the ideas and so should be avoided for that reason. When the goal is correctness, the important ideas are the decisions made and by what criteria choices are selected — it matters that the next thing is the next shortest thing, not how that thing was located in a collection of things. When the goal is implementation, the important ideas are the things that affect the running time. If it is commonly known how to do something, you don't need to repeat that — it's easier to recognize "find the next shortest thing" as a "find best" pattern and know how that works than to recognize a description of the inner workings of a loop as a "find best" pattern. Similarly, if there are multiple well-known ways to achieve something with the same running time, there's no need to specify which way is used.