Extensive research has shown that microRNAs (miRNAs) play a crucial role in cancer progression, treatment, and drug resistance. They have been recognized as promising potential therapeutic targets for overcoming drug resistance in cancer treatment. However, limited attention has been paid to predicting the association between miRNAs and drugs by computational methods. Existing approaches typically focus on constructing miRNA-drug interaction graphs, which may result in their performance being limited by interaction density. In this work, we propose a novel deep learning method that integrates sequence and structural information to infer miRNA-drug associations (MDAs), called DLST-MDA. This approach innovates by utilizing attribute information on miRNAs and drugs instead of relying on the commonly used interaction graph information. Specifically, considering the sequence lengths of miRNAs and drugs, DLST-MDA employs multiscale convolutional neural network (CNN) to learn sequence embeddings at different granularity levels from miRNA and drug sequences. Additionally, it leverages the power of graph neural networks to capture structural information from drug molecular graphs, providing a more representational analysis of the drug features. To evaluate DLST-MDA's effectiveness, we manually constructed a benchmark data set for various experiments based on the latest databases. Results indicate that DLST-MDA performs better than other state-of-the-art methods. Furthermore, case studies of three common anticancer drugs can evidence their usefulness in discovering novel MDAs. The data and source code are released at https://github.com/sheng-n/DLST-MDA.