Indexability and whittle index for restless bandit problems involving reset processes

Keqin Liu; Richard Weber; Qing Zhao

doi:10.1109/CDC.2011.6160533

Indexability and whittle index for restless bandit problems involving reset processes

Keqin Liu^*, Richard Weber, Qing Zhao

^*Corresponding author for this work

Department of Financial and Actuarial Mathematics

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

18 Citations (Scopus)

Abstract

We consider a class of restless multi-armed bandit (RMAB) problems, in which the active action resets the stochastic evolution of the system. We obtain the Whittle index in closed-form, showing that it induces a policy that is equivalent to the myopic policy, and that it is optimal for stochastically identical arms. These results find applications in opportunistic spectrum access and supervisory control systems such as anomaly detection and control.

Original language	English
Title of host publication	2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	7690-7696
Number of pages	7
ISBN (Print)	9781612848006
DOIs	https://doi.org/10.1109/CDC.2011.6160533
Publication status	Published - 2011
Event	2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011 - Orlando, FL, United States Duration: 12 Dec 2011 → 15 Dec 2011

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011
Country/Territory	United States
City	Orlando, FL
Period	12/12/11 → 15/12/11

Keywords

myopic policy
Reset processes
restless multi-armed bandit
Whittle index policy

Access to Document

10.1109/CDC.2011.6160533

Cite this

Liu, K., Weber, R., & Zhao, Q. (2011). Indexability and whittle index for restless bandit problems involving reset processes. In 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011 (pp. 7690-7696). Article 6160533 (Proceedings of the IEEE Conference on Decision and Control). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2011.6160533

@inproceedings{32dafb3d05854782bfa8d5615d494814,

title = "Indexability and whittle index for restless bandit problems involving reset processes",

abstract = "We consider a class of restless multi-armed bandit (RMAB) problems, in which the active action resets the stochastic evolution of the system. We obtain the Whittle index in closed-form, showing that it induces a policy that is equivalent to the myopic policy, and that it is optimal for stochastically identical arms. These results find applications in opportunistic spectrum access and supervisory control systems such as anomaly detection and control.",

keywords = "myopic policy, Reset processes, restless multi-armed bandit, Whittle index policy",

author = "Keqin Liu and Richard Weber and Qing Zhao",

year = "2011",

doi = "10.1109/CDC.2011.6160533",

language = "English",

isbn = "9781612848006",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "7690--7696",

booktitle = "2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011",

note = "2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011 ; Conference date: 12-12-2011 Through 15-12-2011",

}

Liu, K, Weber, R & Zhao, Q 2011, Indexability and whittle index for restless bandit problems involving reset processes. in 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011., 6160533, Proceedings of the IEEE Conference on Decision and Control, Institute of Electrical and Electronics Engineers Inc., pp. 7690-7696, 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011, Orlando, FL, United States, 12/12/11. https://doi.org/10.1109/CDC.2011.6160533

Indexability and whittle index for restless bandit problems involving reset processes. / Liu, Keqin; Weber, Richard; Zhao, Qing.
2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011. Institute of Electrical and Electronics Engineers Inc., 2011. p. 7690-7696 6160533 (Proceedings of the IEEE Conference on Decision and Control).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Indexability and whittle index for restless bandit problems involving reset processes

AU - Liu, Keqin

AU - Weber, Richard

AU - Zhao, Qing

PY - 2011

Y1 - 2011

N2 - We consider a class of restless multi-armed bandit (RMAB) problems, in which the active action resets the stochastic evolution of the system. We obtain the Whittle index in closed-form, showing that it induces a policy that is equivalent to the myopic policy, and that it is optimal for stochastically identical arms. These results find applications in opportunistic spectrum access and supervisory control systems such as anomaly detection and control.

AB - We consider a class of restless multi-armed bandit (RMAB) problems, in which the active action resets the stochastic evolution of the system. We obtain the Whittle index in closed-form, showing that it induces a policy that is equivalent to the myopic policy, and that it is optimal for stochastically identical arms. These results find applications in opportunistic spectrum access and supervisory control systems such as anomaly detection and control.

KW - myopic policy

KW - Reset processes

KW - restless multi-armed bandit

KW - Whittle index policy

UR - http://www.scopus.com/inward/record.url?scp=84860679908&partnerID=8YFLogxK

U2 - 10.1109/CDC.2011.6160533

DO - 10.1109/CDC.2011.6160533

M3 - Conference Proceeding

AN - SCOPUS:84860679908

SN - 9781612848006

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 7690

EP - 7696

BT - 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011

Y2 - 12 December 2011 through 15 December 2011

ER -

Liu K, Weber R, Zhao Q. Indexability and whittle index for restless bandit problems involving reset processes. In 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011. Institute of Electrical and Electronics Engineers Inc. 2011. p. 7690-7696. 6160533. (Proceedings of the IEEE Conference on Decision and Control). doi: 10.1109/CDC.2011.6160533

Indexability and whittle index for restless bandit problems involving reset processes

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this