14b - Comparing Optimizers

of 2
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  14b-1/2 CHE 5703 FALL 2014 Rhinehart Comparing Optimizers –  or any options Which optimizer is best? Here the objective function is to choose the algorithm that minimizes all of the bad aspects and maximizes all of the good aspects. Your DV is the optimizer type. This is optimizing the choice of optimizer. Min J = undesirable aspects  –  desired aspects {optimizer choices} A question is, “How to get measures of the OF aspects?” Here we need to work in Bloom’s Analysis Cognitive Level. Suppose that you decide that algorithm complexity is a bad thing, because simplicity is a good thing. K.I.S.S. is a fundamental value for engineering practice. But, complexity is an abstract concept. You might rank from simplest to worst R3 Heuristic, Golden Section, Secant, Newton’s, Successive quadratic. The ranking 1 st , 2 nd , 3 rd , etc. would be your personal assessment (the numerical values, metrics, provide a measure of the abstract concept). But any assessment should have a defensible basis and a universally verifiable value. For complexity, appropriate metrics might be lines of code, education level required to understand the algorithm, number of user inputs, or number of conditional statements. The problem with a rank, such as 1 st , 2 nd , 3 rd , etc. is that 1 st  and 2 nd  may be very close and 3 rd  may be a distant 3 rd . Rank does not provide relative position. Further, if you learned Newton’s first, because it is familiar to you, you may think it simpler than Golden Section. Rank is OK to use as a measure of goodness, but it is not a universal specific truth. Suppose you also decide that computer work is a bad thing. Work is an abstract concept of badness. The number of iterations to converge is a metric that you could use to assess work. However, some optimizers need more function evaluations per iteration than others, so perhaps number of function evaluations (NOFE) to converge could be a better metric to use as the assessment of work. But, successive quadratic, in the form of my algorithm in the line search examples, only uses one function evaluation at each iteration, but it does all of the computer work to solve for the a, b, and c, coefficients in the linear equation set. So, perhaps execution time is a better measure of work than NOFE. But computer execution time depends on the computer (core speed and how it handles particular operations between devices), on the operating system (how it manages operations), and on the algorithm programmer (how much display overhead is integrated in the basic code). So, perhaps a count of the essential read-write, get-put, and mathematical operations is a better measure of work than either NOFE or execution time. Let’s say you choose NOFE.  Whatever you decide as a measure of “work”, you do trials to get that metric for each DV (optimizer).  Suppose you decide that probability of finding the global optimum is a good thing. There are two issues with this. First, within multi-optima the probability of finding the global optimum is a “yes you did” or “no that was not the global.” But, second, within constraints , or poor surface characterization or flat spots, etc., the optimizer might stop in the vicinity of the global but not exactly on it. You could run each optimizer 1000 times, chose the best result from all optimizers to represent perfection, and then count the number of results for each optimizer that are in the vicinity of the ultimate , and of those measure the variance from “perfection” . The initial trial solution (starting point) will define where the algorithm goes and how many function evaluations it needs to converge. One algorithm might converge rapidly from one point, but  14b-2/2 CHE 5703 FALL 2014 Rhinehart not another because of local surface features. However, the other optimizer might be designed to cope with the features and have an alternate record. So random initialization might be a good way to see the breadth of responses, but replicate trials from random initializations will not produce exact duplicate results. So, statistical comparisons need to be made on replicate results, such as a t-test of differences in NOFE. And the number of trials needs to be high enough to permit differences to be statistically significant. Further, one optimizer (Newton’s, successive quadratic) might jump to the solution in one step if the function is a simple quadratic response, but it might become hopelessly lost on a different function. How do you compare? You need to choose a set of test functions that represent a diversity of features. You could combine several OF aspects to create one overall OF: min (NOFE + Rank - fraction near ultimate). But, how would you scale complexity (rank) with NOFE with fraction? You could use the EC approach. Defend your choice of EC values. Alternately, you could plot each DV (optimizer choice) result on an NOFE vs. Rank graph, and then see if any optimizers dominate the others. See which are Pareto Optimal. Whatever you do to evaluate best, the outcome will depend on: 1) Your interpretation best, your concepts of goodness or badness, desirability or undesirability. 2) Your choice of metrics to assess that aspect. 3) Your choice of experiments to gather assessment data (Only use function 12? Use ten functions to cover an inclusive range of surface aberrations? Only evaluate each function once? Run the optimizer 10,000 times from random initializations from a range of parameter values for each function to be able to see the truth through the statistical vagaries?) 4) Your choice of parameters (expansion coefficients, attenuation factors, starting range, number of players, convergence criteria, number of replicate trials, EC values, etc. for each algorithm. And 5) How you decide to combine and interpret the assessment data. Life is like that. Personal and business decisions are like that. To be appreciated, to be valued, to have your recommendations accepted, you need to be sure that all stakeholders accept your choices in the five steps above (agree that your analysis is fair, comprehensive, complete, not infected by chance or personal bias). So, you need to reveal choices in Steps 1-5. For the assignment, I’ll accept any of any number of choices for steps 1), 2), 3), 4), and 5), except choices representing a trivial, token analysis. But don’t be excessive either, don’t strive for perfection in completeness of comprehensiveness. Balance perfection with sufficiency.
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks