On the myopic policy for a class of restless bandit problems with applications in dynamic multichannel access

Keqin Liu*, Qing Zhao

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

3 Citations (Scopus)

Abstract

We consider a class of restless multi-armed bandit problems that arises in multi-channel opportunistic communications, where channels are modeled as independent and stochastically identical Gilbert-Elliot channels and channel state observations are subject to errors. We show that the myopic channel selection policy has a semi-universal structure that obviates the need to know the Markovian transition probabilities of the channel states. Based on this structure, we establish closed-form lower and upper bounds on the steady-state throughput achieved by the myopic policy. Furthermore, we characterize the approximation factor of the myopic policy to bound its worst-case performance loss with respect to the optimal performance.

Original languageEnglish
Title of host publicationProceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3592-3597
Number of pages6
ISBN (Print)9781424438716
DOIs
Publication statusPublished - 2009
Event48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009 - Shanghai, China
Duration: 15 Dec 200918 Dec 2009

Publication series

NameProceedings of the IEEE Conference on Decision and Control
ISSN (Print)0743-1546
ISSN (Electronic)2576-2370

Conference

Conference48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009
Country/TerritoryChina
CityShanghai
Period15/12/0918/12/09

Keywords

  • Dynamic multi-channel access
  • Myopic policy
  • Restless multi-armed bandit

Fingerprint

Dive into the research topics of 'On the myopic policy for a class of restless bandit problems with applications in dynamic multichannel access'. Together they form a unique fingerprint.

Cite this