Open Collections

UBC Faculty Research and Publications

Best practices for comparing optimization algorithms Beiranvand, Vahid; Hare, Warren; Lucet, Yves

Abstract

Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation. In this paper, we systematically review the benchmarking process of optimization algorithms, and discuss the challenges of fair comparison. We provide suggestions for each step of the comparison process and highlight the pitfalls to avoid when evaluating the performance of optimization algorithms. We also discuss various methods of reporting the benchmarking results. Finally, some suggestions for future research are presented to improve the current benchmarking process.

Item Metadata

Title
Best practices for comparing optimization algorithms
Creator
Beiranvand, Vahid; Hare, Warren; Lucet, Yves
Publisher
Springer
Date Issued
2017
Description
Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation. In this paper, we systematically review the benchmarking process of optimization algorithms, and discuss the challenges of fair comparison. We provide suggestions for each step of the comparison process and highlight the pitfalls to avoid when evaluating the performance of optimization algorithms. We also discuss various methods of reporting the benchmarking results. Finally, some suggestions for future research are presented to improve the current benchmarking process.
Subject
Benchmarking; Algorithm comparison; Guidelines; Performance; Software; Testing; Metric; Timing; Optimization algorithms
Genre
Article
Type
Text
Language
eng
Date Available
2019-07-17
Provider
Vancouver : University of British Columbia Library
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
DOI
10.14288/1.0379904
URI
http://hdl.handle.net/2429/71041
Affiliation
Arts and Sciences, Irving K. Barber School of (Okanagan); Computer Science, Mathematics, Physics and Statistics, Department of (Okanagan)
Citation
Beiranvand, V., Hare, W., & Lucet, Y. (2017). Best practices for comparing optimization algorithms. Optimization and Engineering, 18(4), 815-848.
Publisher DOI
10.1007/s11081-017-9366-1
Peer Review Status
Reviewed
Scholarly Level
Faculty
Rights URI
http://creativecommons.org/licenses/by-nc-nd/4.0/
Aggregated Source Repository
DSpace

Item Media

Item Citations and Data

Rights

Attribution-NonCommercial-NoDerivatives 4.0 International