مليحه حاجي حسيني گزستاني

The Medical Visual Qestion Answering (MVQA) problem is a challenging and newfound problem that uses a combination of natural language processing and computer vision. In this task, the system must correctly answer the questions asked about an input image. This problem is used in various fields. In this thesis, the application of this issue in the field of medicine is discussed, and the first dataset produced in both Persian and English languages for this task is introduced and eva‎luated. Also, some researches about the use of artificial intelligence in medical tasks have been presented. The Medical Visual Qestion Answering (MVQA) problem is a challenging and newfound problem that uses a combination of natural language processing and computer vision. In this task, the system must correctly answer the questions asked about an input image. This problem is used in various fields such as medical, Assistant to the blind people, video surveillance scenarios, education and cultural heritage and advertising. In this thesis, the application of this problem in the field of medicine is discussed. Due to the slow progress of artificial intelligence in medical science, it was necessary to first define and examine simpler problems such as classification problems in the field of medicine so that later they can be developed in the form of visual question answering problems. For this reason, an effort was made to examine similar projects in the field of medicine in parallel. As the first action, the IMAGECLEF World Championships, which have been held since 2003, were identified. Since 2004, the medical section has been added to these competitions, and every year many groups from all over the world compete in this section. In order to gain experience in this field and compete with other students around the world, in 2022, as a representative group from Iran and the University of Science and Technology named IUST_NLPLAB, we competed in this course of competition and were able to rank in the medical image captioning section. get the first gaining this experience was able to provide a good basis for solving the next problems of artificial intelligence in the medical problems. In the next part, an attempt was made to use the capacity of joint research work with industry or other universities in the field of medicine. For this purpose, a collaboration with Tehran University of Medical Sciences was carried out for the automatic detection of skin disease patterns, which in the first stage was able to achieve good and acceptable results and lay the foundation for continued cooperation to diagnose more diseases. Also, in line with the issue of medical imaging questions and answers, the first Persian-English bilingual dataset of this field, which includes organs, imaging modalities and abnormalities, was produced and eva‎luated, which can be the basis for further research on this issue, especially in persian language, and the possibility provide the development of practical software to help patients and doctors.

ﯾﺎﺩﮔﯿﺮﯼ ﻋﻤﯿﻖ , ﭘﺮﺩﺍﺯﺵ ﺯﺑﺎﻥ ﻃﺒﯿﻌي , ﺑﯿﻨﺎﯾﯽ ﻣﺎﺷﯿﻦ , ﭘﺮﺳﺶ ﻭ ﭘﺎﺳﺦ ﺗﺼﻮﯾﺮﯼ

Visual Question Answering , Deep Learning , Natural Language Processing , Computer Vision

Ⅿaⅼihe Hajihosseini Gazestani

Dr.Sayyed Sauleh Eetemadi