TY - GEN
T1 - A restless bandit formulation of opportunistic access
T2 - 5th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks Workshops, SECON 2008
AU - Liu, Keqin
AU - Zhao, Qing
PY - 2008
Y1 - 2008
N2 - We focus on an opportunistic communication system consisting of multiple independent channels with time-varying states. With limited sensing, a user can only sense and access a subset of channels and accrue rewards determined by the state of the sensed channels. We formulate the problem of optimal sequential channel probing as a restless multi-armed bandit process, for which a powerful index policy-Whittle's index policy-can be implemented based on the indexability of the system. Exploiting the underlying structure of the multi-channel opportunistic access problem, we establish the indexability and obtain the Whittle's index in closed-form, which leads to a direct implementation of Whittle's index policy with little complexity. Furthermore, we show that Whittle's index policy is equivalent to the myopic policy when channels are statistically identical.
AB - We focus on an opportunistic communication system consisting of multiple independent channels with time-varying states. With limited sensing, a user can only sense and access a subset of channels and accrue rewards determined by the state of the sensed channels. We formulate the problem of optimal sequential channel probing as a restless multi-armed bandit process, for which a powerful index policy-Whittle's index policy-can be implemented based on the indexability of the system. Exploiting the underlying structure of the multi-channel opportunistic access problem, we establish the indexability and obtain the Whittle's index in closed-form, which leads to a direct implementation of Whittle's index policy with little complexity. Furthermore, we show that Whittle's index policy is equivalent to the myopic policy when channels are statistically identical.
KW - Indexability
KW - Opportunistic access
KW - Optimal channel probing
KW - Restless multi-armed bandit
KW - Whittle's index policy
UR - http://www.scopus.com/inward/record.url?scp=52049120549&partnerID=8YFLogxK
U2 - 10.1109/SAHCNW.2008.12
DO - 10.1109/SAHCNW.2008.12
M3 - Conference Proceeding
AN - SCOPUS:52049120549
SN - 9781424425631
T3 - 2008 5th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks Workshops, SECON
BT - 2008 5th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks Workshops, SECON
Y2 - 16 June 2008 through 20 June 2008
ER -