This paper presents an approach to web API recommendation for mashup development using reinforcement learning (RL). Specifically, we present a RL approach, capable of adapting to the dynamic nature of web API quality properties to recommend web APIs for optimal mashup solution. The approach is also capable of recommending replacement web APIs to existing mashups in a dynamic environment, where the quality properties of the component web APIs continue to change. Since it is challenging to obtain quality of service parameters, our approach models mashup reward using external quality factors of web APIs, which drives the evaluation of its suitability for integration into a mashup application.