Python数据处理
¶

11. Python并行计算
¶

主讲人:丁平尖

Python并行计算¶

  • 多线程(Multithreading):
    • Python 中的多线程是指在同一个进程中创建多个线程,每个线程执行不同的任务。由于 Python 的全局解释器锁(GIL),多线程在 CPU 密集型任务中并不能真正实现并行执行。
  • 多进程(Multiprocessing):
    • Python 中的多进程是指创建多个进程,每个进程执行不同的任务。由于每个进程有自己的内存空间,多进程可以真正实现并行执行。

CPU 密集型任务和 IO 密集型任务¶

  • CPU密集型
    • CPU密集型也叫计算密集型,是指I/O在很短时间就可以完成,CPU需要大量的计算和处理,特点是CPU占用率相当高
    • 例如:压缩解压缩,加密解密,正则表达式搜索
  • IO密集型
    • IO密集型指的是系统运行大部分的状况是CPU在等IO的读写操作,CPU占用率很低
    • 例如:文件处理程序,网络爬虫程序,读写数据库程序

多进程和多线程的对比¶

  • 多进程Process(multiprocessing):
    • 可以利用多核CPU并行运算
    • 占用资源多,可启动数目比线程少
    • 适用于:CPU密集型任务
    • https://docs.python.org/3/library/multiprocessing.html
  • 多线程Thread(threading):
    • 相比进程,更加轻量级,占用资源少
    • 相比进程,多线程只能并行执行,不能利用多CPU
    • 适用于:IO密集型任务
    • https://docs.python.org/3/library/threading.html

Python速度慢的原因¶

  • 对性能要求高的模块不会使用python开发
  • Python速度慢的原因
    • 动态类型语言,随时检查类型
    • 边解释边执行
    • GIL(全局解释器锁):无法利用多核CPU并发执行

全局解释器锁GIL¶

  • 全局解释器锁(Global Interpreter Lock,缩写GIL)
  • 是程序设计语言解释器用于同步线程的一种机制,它使得任何时刻仅有一个线程在执行。即便在多核心处理器尚,使用GIL的解释器也只允许同一时间执行一个线程。
  • http://www.dabeaz.com/python/UnderstandingGIL.pdf
    • When a thread is running, it holds the GIL
    • GIL released on I/O (read,write,send,recv,etc.)

image.png

Python创建多线程的方法¶

  • 怎么创建一个线程
    • t = threading.Thread(target=my_func, args=())
  • 启动线程
    • t.start()
  • 等待结束
    • t.join()
    • 当你创建一个线程并启动它时,主线程会继续执行,而新创建的线程会在后台运行。如果你不使用 join() 方法,主线程可能会在新线程结束之前就退出这会导致以下问题:
      • 资源泄漏:如果主线程退出,而新线程还在运行,新线程可能会占用系统资源,导致资源泄漏。
      • 数据不一致:如果主线程退出,而新线程还在运行,新线程可能会修改共享数据,而主线程已经退出,导致数据不一致。
In [1]:
import threading
import time
def new_thread():
    time.sleep(0.1)
    print("new thread")

t = threading.Thread(target=new_thread)
t.start()
# t.join()
print("main thread")
main thread
new thread
In [2]:
t = threading.Thread(target=new_thread)
t.start()
t.join()
print("main thread")
new thread
main thread
In [3]:
import urllib.request
urls = [f"https://pubmed.ncbi.nlm.nih.gov/?term=machine%20learning&page={page}" for page in range(1, 50+1)]

def craw(url):
    r = urllib.request.urlopen(url)
    return(r.read())

urls[0], len(craw(urls[0]))
Out[3]:
('https://pubmed.ncbi.nlm.nih.gov/?term=machine%20learning&page=1', 282652)
In [4]:
def single_thread():
    for url in urls:
        craw(url)

import threading
def multi_thread():
    threads = [threading.Thread(target=craw, args=(url,)) for url in urls]
    for t in threads:
        t.start()
    for t in threads:
        t.join()

import time
start = time.time()
single_thread()
end = time.time()
print("single thread cost", end - start)
start = time.time()
multi_thread()
end = time.time()
print("multi thread cost", end - start)
single thread cost 57.911908864974976
multi thread cost 2.104243755340576

生产者消费者爬虫的架构¶

  • 配置不同的资源,如线程数

image.png

多线程数据通信的queue.Queue¶

  • queue.Queue可以用于多线程之间的,线程安全的数据通信
  • 导入类库:import queue
  • 创建Queue
    • q = queue.Queue()
  • 添加元素
    • q.put(item)
  • 获取元素
    • item = q.get()
  • 查询状态
    • 查看元素的多少:q.qsize()
    • 判断是否为空:q.empty()
    • 判断是否已满:q.full()
In [5]:
import re
from bs4 import BeautifulSoup
def craw(url):
    r = urllib.request.urlopen(url)
    return r.read()

def parse(html):
    soup = BeautifulSoup(html, 'html.parser')
    links = soup.find_all('a', attrs={"class": "docsum-title"})
    return [(link['href'], re.sub(r'^[\s]+|\B\s+\B', '', link.text)) for link in links]
In [6]:
html = craw(urls[0])
links = parse(html)
links
Out[6]:
[('/34518686/', 'A guide to machine learning for biologists.'),
 ('/26572668/', 'Machine Learning in Medicine.'),
 ('/32704420/',
  'Introduction to Machine Learning, Neural Networks, and Deep Learning.'),
 ('/30102808/', 'eDoctor: machine learning and the future of medicine.'),
 ('/32800297/', 'Supervised Machine Learning: A Brief Primer.'),
 ('/34338485/', 'Machine learning for cardiology.'),
 ('/33290932/',
  'Machine learning model for predicting malaria using clinical information.'),
 ('/32011262/',
  'A Review on Machine Learning for EEG Signal Processing in Bioengineering.'),
 ('/31539636/',
  'Machine learning for clinical decision support in infectious diseases: a narrative review of current applications.'),
 ('/31818379/', 'Machine Learning Principles for Radiology Investigators.')]
In [7]:
import queue
import threading
# 生产者
def do_craw(url_queue: queue.Queue, html_queue: queue.Queue):
    while True: 
        url = url_queue.get()
        html = craw(url)
        html_queue.put(html)

def do_parse(html_queue: queue.Queue, fout):
    while True:
        html = html_queue.get()
        results = parse(html)
        for result in results:
            fout.write(str(result) + '\n')
In [8]:
url_queue = queue.Queue()
html_queue = queue.Queue()

for url in urls:
    url_queue.put(url)

for idx in range(3):
    t = threading.Thread(target=do_craw, args=(url_queue, html_queue))
    t.start()

fout = open("tmp.txt", "w")
for idx in range(2):
    t = threading.Thread(target=do_parse, args=(html_queue, fout))
    t.start()
In [32]:
!cat tmp.txt
('/25692180/', 'Machine learning for medical applications.')
('/35953649/', 'Self-supervised learning in medicine and healthcare.')
('/34450960/', 'Machine Learning Enhances the Performance of Bioreceptor-Free Biosensors.')
('/34585185/', 'Machine learning and chemometrics for electrochemical sensors: moving forward to the future of analytical chemistry.')
('/34731480/', 'Machine Learning from Omics Data.')
('/36124599/', 'Machine fault detection methods based on machine learning algorithms: A review.')
('/33826964/', 'Basic of machine learning and deep learning in imaging for medical physicists.')
('/34302331/', 'The Utility of Unsupervised Machine Learning in Anatomic Pathology.')
('/38565358/', 'Applications of machine learning in phylogenetics.')
('/34203119/', 'An Overview of Machine Learning within Embedded and Mobile Devices-Optimizations and Applications.')
('/34518686/', 'A guide to machine learning for biologists.')
('/26572668/', 'Machine Learning in Medicine.')
('/32704420/', 'Introduction to Machine Learning, Neural Networks, and Deep Learning.')
('/30102808/', 'eDoctor: machine learning and the future of medicine.')
('/32800297/', 'Supervised Machine Learning: A Brief Primer.')
('/34338485/', 'Machine learning for cardiology.')
('/33290932/', 'Machine learning model for predicting malaria using clinical information.')
('/32011262/', 'A Review on Machine Learning for EEG Signal Processing in Bioengineering.')
('/31539636/', 'Machine learning for clinical decision support in infectious diseases: a narrative review of current applications.')
('/31818379/', 'Machine Learning Principles for Radiology Investigators.')
('/32449232/', 'The use of artificial intelligence, machine learning and deep learning in oncologic histopathology.')
('/33185417/', 'Advancing Biosensors with Machine Learning.')
('/38079572/', 'Evaluating Machine Learning Methods of Analyzing Multiclass Metabolomics.')
('/36222893/', 'Supervised machine learning and associated algorithms: applications in orthopedic surgery.')
('/33036008/', 'Supervised machine learning tools: a tutorial for clinicians.')
('/34392886/', 'Machine Learning, Deep Learning, and Closed Loop Devices-Anesthesia Delivery.')
('/30609102/', 'Machine learning in suicide science: Applications and ethics.')
('/38057999/', 'Review of machine learning and deep learning models for toxicity prediction.')
('/32645448/', 'Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition.')
('/32245523/', 'Machine learning and clinical epigenetics: a review of challenges for diagnosis and classification.')
('/33316137/', 'Machine learning for predictive data analytics in medicine: A review illustrated by cardiovascular and nuclear medicine examples.')
('/30971806/', 'Deep learning: new computational modelling techniques for genomics.')
('/35830864/', 'Machine-designed biotherapeutics: opportunities, feasibility and advantages of deep learning in computational antibody discovery.')
('/34537858/', 'Radiomics, machine learning, and artificial intelligence-what the neuroradiologist needs to know.')
('/32456690/', 'How machine learning could be used in clinical practice during an epidemic.')
('/30903692/', 'Simulation-assisted machine learning.')
('/33840471/', 'Deus ex machina? Demystifying rather than deifying machine learning.')
('/32517778/', 'The use of machine learning in rare diseases: a scoping review.')
('/34092975/', 'State of machine and deep learning in histopathological applications in digestive diseases.')
('/37177382/', 'A Comprehensive Review on Machine Learning in Healthcare Industry: Classification, Restrictions, Opportunities and Challenges.')
('/32804360/', 'Machine Learning for Biomedical Time Series Classification: From Shapelets to Deep Learning.')
('/35026457/', 'Uncharted Waters of Machine and Deep Learning for Surgical Phase Recognition in Neurosurgery.')
('/37080758/', 'Using traditional machine learning and deep learning methods for on- and off-target prediction in CRISPR/Cas9: a review.')
('/34731478/', 'Artificial Intelligence, Machine Learning, and Deep Learning in Real-Life Drug Design Cases.')
('/38966252/', 'Editorial: Machine learning and deep learning applications in pathogenic microbiome research.')
('/36433236/', 'Machine Learning for Industry 4.0: A Systematic Review Using Deep Learning-Based Topic Modelling.')
('/36597194/', 'Machine Learning Assisted Clustering of Nanoparticle Structures.')
('/28294138/', 'Machine learning applications in cell image analysis.')
('/30071556/', 'Machine learning and deep learning applied in ultrasound.')
('/33891263/', 'Machine learning-based modeling to predict inhibitors of acetylcholinesterase.')
('/34967861/', 'Machine Learning-Based Anomaly Detection Techniques in Ophthalmology.')
('/38383870/', 'A Review of Machine Learning Algorithms for Biomedical Applications.')
('/38156410/', 'Machine learning (ML) techniques as effective methods for evaluating hair and skin assessments: A systematic review.')
('/37567408/', 'Advances and applications of machine learning and deep learning in environmental ecology and health.')
('/35933043/', 'A systematic review on machine learning and deep learning techniques in cancer survival prediction.')
('/35253294/', 'Special issue on machine learning and deep learning in magnetic resonance.')
('/38058114/', 'From pixels to insights: Machine learning and deep learning for bioimage analysis.')
('/26581149/', "Machine Learning Approaches for Predicting Radiation Therapy Outcomes: A Clinician's Perspective.")
('/36416869/', 'Computer-aided detection and diagnosis/radiomics/machine learning/deep learning in medical imaging.')
('/31524704/', 'Machine Learning to Decode the Electroencephalography for Post Cardiac Arrest Neuroprognostication.')
('/35821601/', 'Unsupervised and semi-supervised learning: the next frontier in machine learning for plant systems biology.')
('/36085353/', 'Machine learning in project analytics: a data-driven framework and case study.')
('/30100821/', 'Machine learning: supervised methods.')
('/37552450/', 'A survey on advanced machine learning and deep learning techniques assisting in renewable energy generation.')
('/31631639/', '[A review of machine learning in tumor radiotherapy].')
('/35500354/', 'Analysis and comparison of machine learning methods for blood identification using single-cell laser tweezer Raman spectroscopy.')
('/38262145/', 'Can machine learning predict cardiac risk using mammography?')
('/38032968/', 'MLpronto: A tool for democratizing machine learning.')
('/35891045/', 'Machine Learning Methods for Hypercholesterolemia Long-Term Risk Prediction.')
('/33431591/', 'What machine learning can do for developmental biology.')
('/38847379/', 'Brain Disorder Detection and Diagnosis using Machine Learning and Deep Learning - A Bibliometric Analysis.')
('/33916850/', 'A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning.')
('/34545817/', 'Towards Interpretable Machine Learning in EEG Analysis.')
('/31427808/', 'Do no harm: a roadmap for responsible machine learning for health care.')
('/37420672/', 'Terrain Characterization via Machine vs. Deep Learning Using Remote Sensing.')
('/34814265/', 'Exploration of machine algorithms based on deep learning model and feature extraction.')
('/29934890/', 'Machine Learning Methods in Computational Toxicology.')
('/35161928/', 'Multi-Label Active Learning-Based Machine Learning Model for Heart Disease Prediction.')
('/34837786/', 'Machine learning and deep learning enabled fuel sooting tendency prediction from molecular structure.')
('/37087255/', 'Machine learning and deep learning based on the small FT-MIR dataset for fine-grained sampling site recognition of boletus tomentipes.')
('/36460741/', 'Application of machine learning in the diagnosis of vestibular disease.')
('/35743052/', 'Editorial of Special Issue "Deep Learning and Machine Learning in Bioinformatics".')
('/35141903/', 'Hybrid machine learning classification scheme for speaker identification.')

线程锁¶

In [18]:
import threading
import time
import logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

class Account:
    def __init__(self, balance):
        self.balance = balance

def draw(account, draw_amount):
    if account.balance >= draw_amount:
        time.sleep(0.1)
        logging.info(f"{threading.current_thread().name}: 取钱成功")
        account.balance -= draw_amount
        logging.info(f"{threading.current_thread().name}: 余额: {account.balance}")
    else:
        logging.info(f"{threading.current_thread().name}: 取钱失败,余额不足")
In [19]:
account = Account(1000)
ta = threading.Thread(name="ta", target=draw, args=(account, 800))
tb = threading.Thread(name="tb", target=draw, args=(account, 800))

ta.start()
tb.start()
2024-11-06 09:33:23,545 - INFO - ta: 取钱成功
2024-11-06 09:33:23,546 - INFO - tb: 取钱成功
2024-11-06 09:33:23,546 - INFO - ta: 余额: 200
2024-11-06 09:33:23,547 - INFO - tb: 余额: -600
In [20]:
import threading
import time

lock = threading.Lock()

class Account:
    def __init__(self, balance):
        self.balance = balance

def draw(account, draw_amount):
    with lock:
        if account.balance >= draw_amount:
            time.sleep(0.1)
            logging.info(f"{threading.current_thread().name}, 取钱成功")
            account.balance -= draw_amount
            logging.info(f"{threading.current_thread().name},余额: {account.balance}")
        else:
            logging.info(f"{threading.current_thread().name}, 取钱失败,余额不足")
In [21]:
account = Account(1000)
ta = threading.Thread(name="ta", target=draw, args=(account, 800))
tb = threading.Thread(name="tb", target=draw, args=(account, 800))

ta.start()
tb.start()
2024-11-06 09:34:12,429 - INFO - ta, 取钱成功
2024-11-06 09:34:12,430 - INFO - ta,余额: 200
2024-11-06 09:34:12,430 - INFO - tb, 取钱失败,余额不足

线程池¶

image-2.png

  • 新建线程系统需要分配资源,终止线程系统需要回收资源,如果可以重用线程,则可以减去新建/终止的开销

image.png

使用线程池的好处¶

  • 提升性能:因为减去大量新建,终止线程的开销,重用了线程资源
  • 适用场景:适合处理突发性大量请求或需要大量线程完成任务,但实际任务处理时间较短
  • 防御功能:能有效避免系统因为创建线程过多,而导致负荷过大响应变慢等问题
  • 代码优势:使用线程池的语法比自己新建线程执行线程更加简洁

threading中的线程池: ThreadPoolExecutor¶

In [12]:
from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor() as executor:
    print(executor._max_workers)
    htmls = executor.map(craw, urls)

with ThreadPoolExecutor() as pool:
    results = pool.map(parse, htmls)

    for result in results:
        print(result)
20
[('/34518686/', 'A guide to machine learning for biologists.'), ('/26572668/', 'Machine Learning in Medicine.'), ('/32704420/', 'Introduction to Machine Learning, Neural Networks, and Deep Learning.'), ('/30102808/', 'eDoctor: machine learning and the future of medicine.'), ('/32800297/', 'Supervised Machine Learning: A Brief Primer.'), ('/34338485/', 'Machine learning for cardiology.'), ('/33290932/', 'Machine learning model for predicting malaria using clinical information.'), ('/32011262/', 'A Review on Machine Learning for EEG Signal Processing in Bioengineering.'), ('/31539636/', 'Machine learning for clinical decision support in infectious diseases: a narrative review of current applications.'), ('/31818379/', 'Machine Learning Principles for Radiology Investigators.')]
[('/32449232/', 'The use of artificial intelligence, machine learning and deep learning in oncologic histopathology.'), ('/33185417/', 'Advancing Biosensors with Machine Learning.'), ('/38079572/', 'Evaluating Machine Learning Methods of Analyzing Multiclass Metabolomics.'), ('/36222893/', 'Supervised machine learning and associated algorithms: applications in orthopedic surgery.'), ('/33036008/', 'Supervised machine learning tools: a tutorial for clinicians.'), ('/34392886/', 'Machine Learning, Deep Learning, and Closed Loop Devices-Anesthesia Delivery.'), ('/30609102/', 'Machine learning in suicide science: Applications and ethics.'), ('/38057999/', 'Review of machine learning and deep learning models for toxicity prediction.'), ('/32645448/', 'Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition.'), ('/32245523/', 'Machine learning and clinical epigenetics: a review of challenges for diagnosis and classification.')]
[('/25692180/', 'Machine learning for medical applications.'), ('/35953649/', 'Self-supervised learning in medicine and healthcare.'), ('/34450960/', 'Machine Learning Enhances the Performance of Bioreceptor-Free Biosensors.'), ('/34585185/', 'Machine learning and chemometrics for electrochemical sensors: moving forward to the future of analytical chemistry.'), ('/34731480/', 'Machine Learning from Omics Data.'), ('/36124599/', 'Machine fault detection methods based on machine learning algorithms: A review.'), ('/33826964/', 'Basic of machine learning and deep learning in imaging for medical physicists.'), ('/34302331/', 'The Utility of Unsupervised Machine Learning in Anatomic Pathology.'), ('/38565358/', 'Applications of machine learning in phylogenetics.'), ('/34203119/', 'An Overview of Machine Learning within Embedded and Mobile Devices-Optimizations and Applications.')]
[('/33316137/', 'Machine learning for predictive data analytics in medicine: A review illustrated by cardiovascular and nuclear medicine examples.'), ('/30971806/', 'Deep learning: new computational modelling techniques for genomics.'), ('/35830864/', 'Machine-designed biotherapeutics: opportunities, feasibility and advantages of deep learning in computational antibody discovery.'), ('/34537858/', 'Radiomics, machine learning, and artificial intelligence-what the neuroradiologist needs to know.'), ('/32456690/', 'How machine learning could be used in clinical practice during an epidemic.'), ('/30903692/', 'Simulation-assisted machine learning.'), ('/33840471/', 'Deus ex machina? Demystifying rather than deifying machine learning.'), ('/32517778/', 'The use of machine learning in rare diseases: a scoping review.'), ('/34092975/', 'State of machine and deep learning in histopathological applications in digestive diseases.'), ('/37177382/', 'A Comprehensive Review on Machine Learning in Healthcare Industry: Classification, Restrictions, Opportunities and Challenges.')]
[('/32804360/', 'Machine Learning for Biomedical Time Series Classification: From Shapelets to Deep Learning.'), ('/35026457/', 'Uncharted Waters of Machine and Deep Learning for Surgical Phase Recognition in Neurosurgery.'), ('/37080758/', 'Using traditional machine learning and deep learning methods for on- and off-target prediction in CRISPR/Cas9: a review.'), ('/34731478/', 'Artificial Intelligence, Machine Learning, and Deep Learning in Real-Life Drug Design Cases.'), ('/38966252/', 'Editorial: Machine learning and deep learning applications in pathogenic microbiome research.'), ('/36433236/', 'Machine Learning for Industry 4.0: A Systematic Review Using Deep Learning-Based Topic Modelling.'), ('/36597194/', 'Machine Learning Assisted Clustering of Nanoparticle Structures.'), ('/28294138/', 'Machine learning applications in cell image analysis.'), ('/30071556/', 'Machine learning and deep learning applied in ultrasound.'), ('/33891263/', 'Machine learning-based modeling to predict inhibitors of acetylcholinesterase.')]
[('/34967861/', 'Machine Learning-Based Anomaly Detection Techniques in Ophthalmology.'), ('/38383870/', 'A Review of Machine Learning Algorithms for Biomedical Applications.'), ('/38156410/', 'Machine learning (ML) techniques as effective methods for evaluating hair and skin assessments: A systematic review.'), ('/37567408/', 'Advances and applications of machine learning and deep learning in environmental ecology and health.'), ('/35933043/', 'A systematic review on machine learning and deep learning techniques in cancer survival prediction.'), ('/35253294/', 'Special issue on machine learning and deep learning in magnetic resonance.'), ('/38058114/', 'From pixels to insights: Machine learning and deep learning for bioimage analysis.'), ('/26581149/', "Machine Learning Approaches for Predicting Radiation Therapy Outcomes: A Clinician's Perspective."), ('/36416869/', 'Computer-aided detection and diagnosis/radiomics/machine learning/deep learning in medical imaging.'), ('/31524704/', 'Machine Learning to Decode the Electroencephalography for Post Cardiac Arrest Neuroprognostication.')]
[('/35821601/', 'Unsupervised and semi-supervised learning: the next frontier in machine learning for plant systems biology.'), ('/36085353/', 'Machine learning in project analytics: a data-driven framework and case study.'), ('/30100821/', 'Machine learning: supervised methods.'), ('/37552450/', 'A survey on advanced machine learning and deep learning techniques assisting in renewable energy generation.'), ('/31631639/', '[A review of machine learning in tumor radiotherapy].'), ('/35500354/', 'Analysis and comparison of machine learning methods for blood identification using single-cell laser tweezer Raman spectroscopy.'), ('/38262145/', 'Can machine learning predict cardiac risk using mammography?'), ('/38032968/', 'MLpronto: A tool for democratizing machine learning.'), ('/35891045/', 'Machine Learning Methods for Hypercholesterolemia Long-Term Risk Prediction.'), ('/33431591/', 'What machine learning can do for developmental biology.')]
[('/36460741/', 'Application of machine learning in the diagnosis of vestibular disease.'), ('/35743052/', 'Editorial of Special Issue "Deep Learning and Machine Learning in Bioinformatics".'), ('/35141903/', 'Hybrid machine learning classification scheme for speaker identification.'), ('/38287813/', 'Machine and Deep Learning: Artificial Intelligence Application in Biotic and Abiotic Stress Management in Plants.'), ('/29348742/', 'Emerging Trends in Machine Learning for Signal Processing.'), ('/34704215/', 'Tracking strategy changes using machine learning classifiers.'), ('/36838652/', 'Comparative Studies on Resampling Techniques in Machine Learning and Deep Learning Models for Drug-Target Interaction Prediction.'), ('/35701721/', 'Supervised machine learning aided behavior classification in pigeons.'), ('/38802973/', 'A scoping review of machine learning for sepsis prediction- feature engineering strategies and model performance: a step towards explainability.'), ('/36303299/', 'A systematic literature review for the prediction of anticancer drug response using various machine-learning and deep-learning techniques.')]
[('/38847379/', 'Brain Disorder Detection and Diagnosis using Machine Learning and Deep Learning - A Bibliometric Analysis.'), ('/33916850/', 'A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning.'), ('/34545817/', 'Towards Interpretable Machine Learning in EEG Analysis.'), ('/31427808/', 'Do no harm: a roadmap for responsible machine learning for health care.'), ('/37420672/', 'Terrain Characterization via Machine vs. Deep Learning Using Remote Sensing.'), ('/34814265/', 'Exploration of machine algorithms based on deep learning model and feature extraction.'), ('/29934890/', 'Machine Learning Methods in Computational Toxicology.'), ('/35161928/', 'Multi-Label Active Learning-Based Machine Learning Model for Heart Disease Prediction.'), ('/34837786/', 'Machine learning and deep learning enabled fuel sooting tendency prediction from molecular structure.'), ('/37087255/', 'Machine learning and deep learning based on the small FT-MIR dataset for fine-grained sampling site recognition of boletus tomentipes.')]
[('/35957366/', 'Machine Learning Models for Enhanced Estimation of Soil Moisture Using Wideband Radar Sensor.'), ('/35126955/', 'Application of Machine Learning in Rheumatic Immune Diseases.'), ('/35140765/', 'Developing Multiagent E-Learning System-Based Machine Learning and Feature Selection Techniques.'), ('/30867605/', 'Machine learning in quantum spaces.'), ('/28653122/', 'Machine Learning: Discovering the Future of Medical Imaging.'), ('/33346034/', 'Machine Learning Feature Selection for Predicting High Concentration Therapeutic Antibody Aggregation.'), ('/33652397/', 'Machine Learning Systems Applied to Health Data and System.'), ('/35852212/', 'Artificial intelligence, machine learning, and deep learning in orthopedic surgery.'), ('/35875749/', 'A Machine Learning and Deep Learning Approach for Recognizing Handwritten Digits.'), ('/34810369/', 'Machine learning for identification of dental implant systems based on shape - A descriptive study.')]
[('/36720809/', 'Applying Machine Learning to Classify the Origins of Gene Duplications.'), ('/35591194/', 'Rainfall Prediction System Using Machine Learning Fusion for Smart Cities.'), ('/34320410/', 'Probing machine-learning classifiers using noise, bubbles, and reverse correlation.'), ('/32978734/', 'Lake water-level fluctuation forecasting using machine learning models: a systematic review.'), ('/37382799/', 'ADis-QSAR: a machine learning model based on biological activity differences of compounds.'), ('/38656054/', 'Machine learning classification based on k-Nearest Neighbors for PolSAR data.'), ('/36399296/', 'A review of recent developments in the application of machine learning in solar thermal collector modelling.'), ('/30596635/', "Better medicine through machine learning: What's real, and what's artificial?"), ('/30629201/', 'Shining Light Into the Black Box of Machine Learning.'), ('/36236374/', 'Prediction Models for Railway Track Geometry Degradation Using Machine Learning Methods: A Review.')]
[('/36080927/', 'Machine Learning and Lexicon Approach to Texts Processing in the Detection of Degrees of Toxicity in Online Discussions.'), ('/36576347/', 'Do You See What I See? Automated IFE Interpretation Using Machine Learning.'), ('/38633744/', 'Exploring the impact of pathogenic microbiome in orthopedic diseases: machine learning and deep learning approaches.'), ('/38810020/', 'PROBAST Assessment of Machine Learning: Reply.'), ('/35808156/', 'Machine Learning Techniques Based on Primary User Emulation Detection in Mobile Cognitive Radio Networks.'), ('/34915570/', 'Machine Learning Identifies Digital Phenotyping Measures Most Relevant to Negative Symptoms in Psychotic Disorders: Implications for Clinical Trials.'), ('/32838979/', 'Bias and ethical considerations in machine learning and the automation of perioperative risk assessment.'), ('/33121910/', 'A primer for understanding radiology articles about machine learning and deep learning.'), ('/33211015/', 'Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study.'), ('/34783028/', 'RootPainter3D: Interactive-machine-learning enables rapid and accurate contouring for radiotherapy.')]
[('/39325602/', 'A Scoping Review of Machine Learning Applied to Peripheral Nerve Interfaces.'), ('/32049317/', 'ColocML: machine learning quantifies co-localization between mass spectrometry images.'), ('/33902993/', 'Skin tear classification using machine learning from digital RGB image.'), ('/34824579/', 'Enterprise Risk Assessment Based on Machine Learning.'), ('/31488674/', 'Machine learning transforms how microstates are sampled.'), ('/35336502/', 'Solving Inverse Electrocardiographic Mapping Using Machine Learning and Deep Learning Frameworks.'), ('/36234792/', 'Objective Supervised Machine Learning-Based Classification and Inference of Biological Neuronal Networks.'), ('/34352535/', 'Machine learning methods for imbalanced data set for prediction of faecal contamination in beach waters.'), ('/38715436/', 'The new paradigm in machine learning - foundation models, large language models and beyond: a primer for physicians.'), ('/39294357/', 'Rage against machine learning driven by profit.')]
[('/31233628/', 'Artificial intelligence, machine learning and deep learning: definitions and differences.'), ('/32761520/', '[On the future of machine learning in anesthesiology].'), ('/33179985/', 'Leveraging machine learning for predicting human body model response in restraint design simulations.'), ('/37682485/', 'A Transferable Machine Learning Framework for Predicting Transcriptional Responses of Genes Across Species.'), ('/34119773/', 'Machine learning classification of origins and varieties of Tetrastigma hemsleyanum using a dual-mode microscopic hyperspectral imager.'), ('/36177311/', 'A Machine Learning-Based Water Potability Prediction Model by Using Synthetic Minority Oversampling Technique and Explainable AI.'), ('/35773925/', 'Customized and Automated Machine Learning-Based Models for Diabetes Type 2 Classification.'), ('/34372395/', 'FEA and Machine Learning Techniques for Hidden Structure Analysis.'), ('/37696181/', 'A fusion framework of deep learning and machine learning for predicting sgRNA cleavage efficiency.'), ('/32881355/', 'Prediction of Promiscuity Cliffs Using Machine Learning.')]
[('/31422572/', 'Machine learning applications to clinical decision support in neurosurgery: an artificial intelligence augmented systematic review.'), ('/33275806/', 'Integrated analysis of machine learning and deep learning in chili pest and disease identification.'), ('/36132548/', 'Investigation of Applying Machine Learning and Hyperparameter Tuned Deep Learning Approaches for Arrhythmia Detection in ECG Images.'), ('/32335226/', 'Building machine learning models without sharing patient data: A simulation-based analysis of distributed learning by ensembling.'), ('/33339777/', 'A Molecular Image-Based Novel Quantitative Structure-Activity Relationship Approach, Deepsnap-Deep Learning and Machine Learning.'), ('/36899578/', 'Applying machine learning techniques to detect the deployment of spatial working memory from the spiking activity of MT neurons.'), ('/36323790/', 'Self-supervised machine learning for live cell imagery segmentation.'), ('/26743974/', "Machine learning on Parkinson's disease? Let's translate into clinical practice."), ('/29790152/', 'Machine learning algorithms for accurate differential diagnosis of lymphocytosis based on cell population data.'), ('/30336670/', 'Adversarial Controls for Scientific Machine Learning.')]
[('/35831342/', 'Comparing machine learning and deep learning regression frameworks for accurate prediction of dielectrophoretic force.'), ('/34073145/', 'Supervised Machine Learning Methods and Hyperspectral Imaging Techniques Jointly Applied for Brain Cancer Classification.'), ('/36240162/', 'Adversarial attacks against supervised machine learning based network intrusion detection systems.'), ('/34530437/', 'Ensemble modeling with machine learning and deep learning to provide interpretable generalized rules for classifying CNS drugs with high prediction power.'), ('/37139570/', 'Deep Learning for Epidemiologists: An Introduction to Neural Networks.'), ('/32418342/', 'The role of machine and deep learning in modern medical physics.'), ('/36503039/', 'Deep Learning in Bioinformatics and Biomedicine.'), ('/28456276/', 'Recent developments in machine learning for medical imaging applications.'), ('/36033770/', 'Predicting difficult airway intubation in thyroid surgery using multiple machine learning and deep learning algorithms.'), ('/35814538/', 'Research on E-Commerce Database Marketing Based on Machine Learning Algorithm.')]
[('/38761300/', 'Machine learning, deep learning and hernia surgery. Are we pushing the limits of abdominal core health? A qualitative systematic review.'), ('/37943766/', 'Machine learning prediction model based on enhanced bat algorithm and support vector machine for slow employment prediction.'), ('/33984864/', 'A New Paradigm of "Real-Time" Stroke Risk Prediction and Integrated Care Management in the Digital Health Era: Innovations Using Machine Learning and Artificial Intelligence Approaches.'), ('/36623998/', 'Deep learning in image-based phenotypic drug discovery.'), ('/34164114/', 'RNAmining: A machine learning stand-alone and web server tool for RNA coding potential prediction.'), ('/34178926/', 'Deep Learning Assisted Neonatal Cry Classification via Support Vector Machine Models.'), ('/39283390/', 'Adaptive Machine Learning Systems in Medicine: The Post-Authorization Phase.'), ('/39047502/', 'Interpretable machine learning for dermatological disease detection: Bridging the gap between accuracy and explainability.'), ('/38176209/', 'A machine learning-based universal outbreak risk prediction tool.'), ('/36146104/', 'Network Meddling Detection Using Machine Learning Empowered with Blockchain Technology.')]
[('/32506658/', 'Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge.'), ('/30307362/', 'Machine and deep learning for sport-specific movement recognition: a systematic review of model development and performance.'), ('/37291435/', 'Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging.'), ('/35569263/', 'Comparison of machine learning approaches for radioisotope identification using NaI(TI) gamma-ray spectrum.'), ('/37055127/', 'Application of machine learning algorithms in thermal images for an automatic classification of lumbar sympathetic blocks.'), ('/36739164/', 'Applications of Deep Learning in Endocrine Neoplasms.'), ('/35308915/', 'Comparing Deep Learning and Conventional Machine Learning Models for Predicting Mental Illness from History of Present Illness Notations.'), ('/35344997/', 'eSPA+: Scalable Entropy-Optimal Machine Learning Classification for Small Data Problems.'), ('/32197580/', 'Standard machine learning approaches outperform deep representation learning on phenotype prediction from transcriptomics data.'), ('/38142162/', 'Deep Learning and Geriatric Mental Health.')]
[('/37571624/', 'Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning.'), ('/37387111/', 'Predicting Overall Survival in METABRIC Cohort Using Machine Learning.'), ('/37515457/', 'Optimization of machine learning classification models for tumor cells based on cell elements heterogeneity with laser-induced breakdown spectroscopy.'), ('/34746985/', 'Performance comparison of deep learning and machine learning methods in determining wetland water areas using EuroSAT dataset.'), ('/34731473/', 'Deep Learning in Structure-Based Drug Design.'), ('/36903654/', 'Ensemble Learning, Deep Learning-Based and Molecular Descriptor-Based Quantitative Structure-Activity Relationships.'), ('/31922318/', 'Editorial for "Top 10 Reviewer Critiques of Radiology Artificial Intelligence (AI) Articles: Qualitative Thematic Analysis of Reviewer Critiques of Machine Learning / Deep Learning Manuscripts Submitted to JMRI".'), ('/35567312/', 'Deep Learning Applications in Surgery: Current Uses and Future Directions.'), ('/38364640/', 'Prediction of Aureococcus anophageffens using machine learning and deep learning.'), ('/35653511/', 'In silico prediction of chemical aquatic toxicity by multiple machine learning and deep learning approaches.')]
[('/35275181/', 'Applications of deep learning in electron microscopy.'), ('/34306056/', 'Prediction of Heart Disease Using a Combination of Machine Learning and Deep Learning.'), ('/30158611/', 'Machine learning improves forecasts of aftershock locations.'), ('/31363197/', 'Three pitfalls to avoid in machine learning.'), ('/30317133/', 'Learning in the machine: Recirculation is random backpropagation.'), ('/34102430/', 'Classification of broiler behaviours using triaxial accelerometer and machine learning.'), ('/35186054/', 'Marine Data Prediction: An Evaluation of Machine Learning, Deep Learning, and Statistical Predictive Models.'), ('/35062084/', 'Using Machine Learning to Improve Personalised Prediction: A Data-Driven Approach to Segment and Stratify Populations for Healthcare.'), ('/34544924/', 'Deep Learning and Its Application to Function Approximation for MR in Medicine: An Overview.'), ('/31439010/', 'Use of machine learning to analyse routinely collected intensive care unit data: a systematic review.')]
[('/34891833/', 'A new machine learning based user-friendly software platform for automatic radiomics modeling and analysis.'), ('/33032774/', 'How Good Is Machine Learning in Predicting All-Cause 30-Day Hospital Readmission? Evidence From Administrative Data.'), ('/38703965/', 'Ensemble machine learning prediction of anaerobic co-digestion of manure and thermally pretreated harvest residues.'), ('/38032993/', 'Analyzing drama metadata through machine learning to gain insights into social information dissemination patterns.'), ('/38154417/', 'Unsupervised learning of mid-level visual representations.'), ('/36016014/', 'Aggregation Strategy on Federated Machine Learning Algorithm for Collaborative Predictive Maintenance.'), ('/37188731/', 'A general model to predict small molecule substrates of enzymes based on machine and deep learning.'), ('/36266530/', 'Hyperspectral imaging for chemicals identification: a human-inspired machine learning approach.'), ('/38662360/', 'Should the Use of Adaptive Machine Learning Systems in Medicine be Classified as Research?'), ('/32167235/', 'Deep learning a boon for biophotonics?')]
[('/36076915/', 'C10Pred: A First Machine Learning Based Tool to Predict C10 Family Cysteine Peptidases Using Sequence-Derived Features.'), ('/35867362/', 'Self-Supervised Learning for Electroencephalography.'), ('/34220999/', 'Improving the Performance of Machine Learning-Based Network Intrusion Detection Systems on the UNSW-NB15 Dataset.'), ('/35297826/', 'Development of advanced machine learning models for analysis of plutonium surrogate optical emission spectra.'), ('/36617095/', 'Application of Machine Learning Methods for an Analysis of E-Nose Multidimensional Signals in Wastewater Treatment.'), ('/37960551/', 'Multi-Cat Monitoring System Based on Concept Drift Adaptive Machine Learning Architecture.'), ('/36052049/', 'Research on a Machine Learning-Based Method for Assessing the Safety State of Historic Buildings.'), ('/32857133/', 'MACHINE LEARNING ALGORITHMS FOR IDENTIFICATION OF ABNORMAL GLOW CURVES AND ASSOCIATED ABNORMALITY IN CaSO4:DY-BASED PERSONNEL MONITORING DOSIMETERS.'), ('/29576545/', 'Research progress in machine learning methods for gene-gene interaction detection.'), ('/35009698/', 'Comparison of Machine Learning and Sentiment Analysis in Detection of Suspicious Online Reviewers on Different Type of Data.')]
[('/39437806/', 'Classification of EEG evoked in 2D and 3D virtual reality: traditional machine learning versus deep learning.'), ('/36642410/', 'Prediction of anticancer peptides based on an ensemble model of deep learning and machine learning using ordinal positional encoding.'), ('/35548095/', 'Painting Classification in Art Teaching under Machine Learning from the Perspective of Emotional Semantic Analysis.'), ('/31813571/', 'Deep learning for risk assessment: all about automatic feature extraction.'), ('/36721325/', 'PyGenePlexus: a Python package for gene discovery using network-based machine learning.'), ('/34833613/', 'Event Detection for Distributed Acoustic Sensing: Combining Knowledge-Based, Classical Machine Learning, and Deep Learning Approaches.'), ('/35161914/', 'Study on Machine Learning Models for Building Resilience Evaluation in Mountainous Area: A Case Study of Banan District, Chongqing, China.'), ('/34884021/', "Vision-Based Driver's Cognitive Load Classification Considering Eye Movement Using Machine Learning and Deep Learning."), ('/32894241/', 'Utility of MemTrax and Machine Learning Modeling in Classification of Mild Cognitive Impairment.'), ('/33418514/', 'Application of machine learning to improve dairy farm management: A systematic literature review.')]
[('/35408249/', 'Data-Driven Fault Diagnosis Techniques: Non-Linear Directional Residual vs. Machine-Learning-Based Methods.'), ('/34502670/', 'A Novel Unsupervised Machine Learning-Based Method for Chatter Detection in the Milling of Thin-Walled Parts.'), ('/38480847/', 'Evaluation metrics and statistical tests for machine learning.'), ('/39176919/', 'Multi-Objective Performance Optimization of Machine Learning Models in Healthcare.'), ('/37934781/', 'Domain-adaptive neural networks improve supervised machine learning based on simulated population genetic data.'), ('/35891005/', 'Well Performance Classification and Prediction: Deep Learning and Machine Learning Long Term Regression Experiments on Oil, Gas, and Water Production.'), ('/38032965/', 'Link prediction and feature relevance in knowledge networks: A machine learning approach.'), ('/38546090/', 'Prediction and Diagnosis of Breast Cancer Using Machine and Modern Deep Learning Models.'), ('/32403049/', 'Exploration of critical care data by using unsupervised machine learning.'), ('/37471188/', 'Beyond Supervised Learning for Pervasive Healthcare.')]
[('/37579795/', 'Commentary on "A systematic review on machine learning and deep learning techniques in cancer survival prediction": Validation of survival methods.'), ('/31290985/', 'Enhancement of Risk Prediction With Machine Learning: Rise of the Machines.'), ('/34814259/', 'Application of supervised machine learning as a method for identifying DEM contact law parameters.'), ('/35184396/', 'The synergism of spatial metabolomics and morphometry improves machine learning-based renal tumour subtype classification.'), ('/31473577/', 'Distributed semi-supervised learning algorithm based on extreme learning machine over networks using event-triggered communication scheme.'), ('/38438821/', 'Shallow and deep learning classifiers in medical image analysis.'), ('/34236667/', 'Deep Learning for Protein-Protein Interaction Site Prediction.'), ('/39437806/', 'Classification of EEG evoked in 2D and 3D virtual reality: traditional machine learning versus deep learning.'), ('/37036491/', 'The Impact of Supervised Learning Methods in Ultralarge High-Throughput Docking.'), ('/35214523/', 'Hyperspectral Image Labeling and Classification Using an Ensemble Semi-Supervised Machine Learning Approach.')]
[('/33879214/', 'Novel criteria to classify ARDS severity using a machine learning approach.'), ('/30526348/', 'With a Little Help from Machine Learning, Precision Radiology Can Be Feasible.'), ('/32952545/', 'Adoption of Machine Learning in Intelligent Terrain Classification of Hyperspectral Remote Sensing Images.'), ('/31481588/', 'Radiomics: Data Are Also Images.'), ('/34283125/', 'Received Signal Strength Fingerprinting-Based Indoor Location Estimation Employing Machine Learning.'), ('/36081435/', 'IOT-Based Medical Informatics Farming System with Predictive Data Analytics Using Supervised Machine Learning Algorithms.'), ('/36502200/', 'A TDD Framework for Automated Monitoring in Internet of Things with Machine Learning.'), ('/36799417/', 'Cancer Detection Based on Medical Image Analysis with the Help of Machine Learning and Deep Learning Techniques: A Systematic Literature Review.'), ('/39283404/', 'Machine Learning-Generated Clinical Data as Collateral Research: A Global Neuroethical Analysis.'), ('/34502786/', 'Enhanced Changeover Detection in Industry 4.0 Environments with Machine Learning.')]
[('/39261891/', 'Cardiovascular disease diagnosis: a holistic approach using the integration of machine learning and deep learning models.'), ('/35335117/', 'Identification of Pharmacophoric Fragments of DYRK1A Inhibitors Using Machine Learning Classification Models.'), ('/38701634/', 'Prediction model of pressure injury occurrence in diabetic patients during ICU hospitalization--XGBoost machine learning model can be interpreted based on SHAP.'), ('/37203612/', 'Comparative Analysis of Electrodermal Activity Decomposition Methods in Emotion Detection Using Machine Learning.'), ('/32790666/', 'Evolution and impact of bias in human and machine learning algorithm interaction.'), ('/37673561/', 'How does the model make predictions? A systematic literature review on the explainability power of machine learning in healthcare.'), ('/38459463/', 'Deep self-supervised machine learning algorithms with a novel feature elimination and selection approaches for blood test-based multi-dimensional health risks classification.'), ('/31786999/', 'The role of artificial intelligence and machine learning in predicting orthopaedic outcomes.'), ('/29743630/', 'A machine learning model with human cognitive biases capable of learning from small and biased datasets.'), ('/35218736/', 'TMPpred: A support vector machine-based thermophilic protein identifier.')]
[('/32380557/', 'A Method to Extract Feature Variables Contributed in Nonlinear Machine Learning Prediction.'), ('/34938423/', 'Comparison of the Meta-Active Machine Learning Model Applied to Biological Data-Driven Experiments with Other Models.'), ('/38326768/', 'Genomic prediction using machine learning: a comparison of the performance of regularized regression, ensemble, instance-based and deep learning methods on synthetic and empirical data.'), ('/36890378/', 'In AI, is bigger always better?'), ('/39282871/', 'Comparison of machine learning models to predict complications of bariatric surgery: A systematic review.'), ('/30440597/', 'Monitoring Lung Mechanics during Mechanical Ventilation using Machine Learning Algorithms.'), ('/35589934/', 'Proactive approach for preamble detection in 5G-NR PRACH using supervised machine learning and ensemble model.'), ('/36514230/', 'Artificial intelligence 101 for veterinary diagnostic imaging.'), ('/37941177/', 'The Use of Kinematic Features in Evaluating Upper Limb Motor Function Learning Progress Based on Machine Learning.'), ('/36052029/', 'Text-Based Emotion Recognition Using Deep Learning Approach.')]
[('/35990144/', 'Research on the Audit Prediction Model of "Special Bonds + PPP" Project based on Machine Learning.'), ('/36502037/', 'The Application of Deep Learning for the Evaluation of User Interfaces.'), ('/32218142/', 'Monitoring Mixing Processes Using Ultrasonic Sensors and Machine Learning.'), ('/33534111/', 'Item response theory as a feature selection and interpretation tool in the context of machine learning.'), ('/39013808/', 'Artificial intelligence in medicine: The rise of machine learning.'), ('/34300498/', 'Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future.'), ('/35591056/', 'Transport and Application Layer DDoS Attacks Detection to IoT Devices by Using Machine Learning and Deep Learning Models.'), ('/36275984/', 'A Crop Growth Prediction Model Using Energy Data Based on Machine Learning in Smart Farms.'), ('/37385080/', 'Finding functional motifs in protein sequences with deep learning and natural language models.'), ('/37936066/', 'Hybrid deep learning approach to improve classification of low-volume high-dimensional data.')]
[('/35403003/', 'Protocol for state-of-health prediction of lithium-ion batteries based on machine learning.'), ('/34372469/', 'Sea Fog Dissipation Prediction in Incheon Port and Haeundae Beach Using Machine Learning and Deep Learning.'), ('/36679504/', 'A Novel System to Increase Yield of Manufacturing Test of an RF Transceiver through Application of Machine Learning.'), ('/39095606/', 'Identifying diseases symptoms and general rules using supervised and unsupervised machine learning.'), ('/38913480/', "Machine Learning Algorithms in the Personalized Modeling of Incapacitated Patients' Decision Making-Is It a Viable Concept?"), ('/36256230/', 'Construction of classification models for pathogenic bacteria based on LIBS combined with different machine learning algorithms.'), ('/36232571/', 'Precision Medicine Approaches with Metabolomics and Artificial Intelligence.'), ('/33845426/', 'Transfer learning for small molecule retention predictions.'), ('/35901215/', 'Informing geometric deep learning with electronic interactions to accelerate quantum chemistry.'), ('/34545282/', 'Application of Various Machine Learning Techniques in Predicting Total Organic Carbon from Well Logs.')]
[('/34824763/', 'Medical and Health Data Classification Method Based on Machine Learning.'), ('/35602638/', 'Exploration of Black Boxes of Supervised Machine Learning Models: A Demonstration on Development of Predictive Heart Risk Score.'), ('/37714041/', 'Protein classification by autofluorescence spectral shape analysis using machine learning.'), ('/36616878/', 'Diagnosis of Operating Conditions of the Electrical Submersible Pump via Machine Learning.'), ('/36086383/', 'Sequential Learning on sEMGs in Short- and Long-term Situations via Self-training Semi-supervised Support Vector Machine.'), ('/36689455/', 'Evaluations on supervised learning methods in the calibration of seven-hole pressure probes.'), ('/28025565/', 'A Machine Learning Approach to Pedestrian Detection for Autonomous Vehicles Using High-Definition 3D Range Data.'), ('/34814272/', '(1)-norm based safe semi-supervised learning.'), ('/33655353/', 'Estimating evapotranspiration by coupling Bayesian model averaging methods with machine learning algorithms.'), ('/28933947/', 'On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products.')]
[('/39098272/', 'Improved medical waste plasma gasification modelling based on implicit knowledge-guided interpretable machine learning.'), ('/35697984/', 'Prediction of black carbon in marine engines and correlation analysis of model characteristics based on multiple machine learning algorithms.'), ('/36797504/', 'Learning Relationships Between Chemical and Physical Stability for Peptide Drug Development.'), ('/35083949/', 'IoT-based automated water pollution treatment using machine learning classifiers.'), ('/36342692/', 'Important feature identification for perceptual sex of point-light walkers using supervised machine learning.'), ('/39408595/', 'PROTA: A Robust Tool for Protamine Prediction Using a Hybrid Approach of Machine Learning and Deep Learning.'), ('/37960371/', 'AI-Assisted Cotton Grading: Active and Semi-Supervised Learning to Reduce the Image-Labelling Burden.'), ('/36081066/', 'Evaluation of Three Feature Dimension Reduction Techniques for Machine Learning-Based Crop Yield Prediction Models.'), ('/35750007/', 'MGLNN: Semi-supervised learning via Multiple Graph Cooperative Learning Neural Networks.'), ('/36029974/', 'Usability of deep learning pipelines for 3D nuclei identification with Stardist and Cellpose.')]
[('/31753908/', 'Comparing context-dependent call sequences employing machine learning methods: an indication of syntactic structure of greater horseshoe bats.'), ('/33259313/', 'Semisupervised Learning via Axiomatic Fuzzy Set Theory and SVM.'), ('/36257069/', 'Surface similarity parameter: A new machine learning loss metric for oscillatory spatio-temporal data.'), ('/34237784/', 'Semi-automated Conversion of Clinical Trial Legacy Data into CDISC SDTM Standards Format Using Supervised Machine Learning.'), ('/33393291/', 'OpenChem: A Deep Learning Toolkit for Computational Chemistry and Drug Design.'), ('/35009725/', 'An Aggregated Mutual Information Based Feature Selection with Machine Learning Methods for Enhancing IoT Botnet Attack Detection.'), ('/38831432/', 'DREAMER: a computational framework to evaluate readiness of datasets for machine learning.'), ('/35694598/', 'Construction of English and American Literature Corpus Based on Machine Learning Algorithm.'), ('/33805937/', 'Hyperspectral Classification of Blood-Like Substances Using Machine Learning Methods Combined with Genetic Algorithms in Transductive and Inductive Scenarios.'), ('/36038620/', 'Semi-supervised classifier guided by discriminator.')]
[('/31070704/', 'A decision-theoretic approach to the evaluation of machine learning algorithms in computational drug discovery.'), ('/37299953/', 'A New NILM System Based on the SFRA Technique and Machine Learning.'), ('/31772370/', 'AI takes on popular Minecraft game in machine-learning contest.'), ('/36928404/', 'GPT-4 is here: what scientists think.'), ('/33381838/', 'Supervised learning on phylogenetically distributed data.'), ('/36169205/', 'Three simple steps to improve the interpretability of EEG-SVM studies.'), ('/35077979/', 'Component spectra extraction and quantitative analysis for preservative mixtures by combining terahertz spectroscopy and machine learning.'), ('/32992108/', 'Robots join the care team: Making healthcare decisions safer with machine learning and robotics.'), ('/35266390/', 'Unified Deep Learning Model for Multitask Reaction Predictions with Explanation.'), ('/33270566/', 'Investigating Deep Learning Based Breast Cancer Subtyping Using Pan-Cancer and Multi-Omic Data.')]
[('/33201830/', 'Deep Learning Versus Traditional Solutions for Group Trajectory Outliers.'), ('/37550581/', 'Deep-learning-based isolation of perturbation-induced variations in single-cell data.'), ('/35528334/', 'Hyperspectral Image Classification: Potentials, Challenges, and Future Directions.'), ('/29396120/', 'Deep Learning in Radiology: Does One Size Fit All?'), ('/37028023/', 'Self-Supervised Learning for Annotation Efficient Biomedical Image Segmentation.'), ('/34640727/', 'Comparing Methods of Feature Extraction of Brain Activities for Octave Illusion Classification Using Machine Learning.'), ('/36697024/', 'Update on the Use of Artificial Intelligence in Hepatobiliary MR Imaging.'), ('/35632290/', 'Beam Offset Detection in Laser Stake Welding of Tee Joints Using Machine Learning and Spectrometer Measurements.'), ('/34795069/', 'Encoding Health Records into Pathway Representations for Deep Learning.'), ('/32802027/', 'Classification of Task-State fMRI Data Based on Circle-EMD and Machine Learning.')]
[('/37960398/', 'On-Device Execution of Deep Learning Models on HoloLens2 for Real-Time Augmented Reality Medical Applications.'), ('/35228648/', 'Feature engineering solution with structured query language analytic functions in detecting electricity frauds using machine learning.'), ('/38767971/', 'What Are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids.'), ('/31943495/', 'Top 10 Reviewer Critiques of Radiology Artificial Intelligence (AI) Articles: Qualitative Thematic Analysis of Reviewer Critiques of Machine Learning/Deep Learning Manuscripts Submitted to JMRI.'), ('/31977216/', 'Boosting Tree-Assisted Multitask Deep Learning for Small Scientific Datasets.'), ('/36041380/', 'DL 101: Basic introduction to deep learning with its application in biomedical related fields.'), ('/37375375/', 'Deep Learning for Identifying Promising Drug Candidates in Drug-Phospholipid Complexes.'), ('/38547803/', 'Distribution-free Bayesian regularized learning framework for semi-supervised learning.'), ('/38448525/', 'Why scientists trust AI too much - and what to do about it.'), ('/31816343/', 'Supervised and unsupervised algorithms for bioinformatics and data science.')]
[('/37369839/', 'Adaptive algorithms: users must be more vigilant.'), ('/30223423/', 'Comparison of medical image classification accuracy among three machine learning methods.'), ('/33961736/', 'Learned Embeddings from Deep Learning to Visualize and Predict Protein Sets.'), ('/36072718/', 'Stock Portfolio Optimization Using a Combined Approach of Multi Objective Grey Wolf Optimizer and Machine Learning Preselection Methods.'), ('/32245258/', 'Deep Learning-Based Methods for Automatic Diagnosis of Skin Lesions.'), ('/34354742/', 'Automated Amharic News Categorization Using Deep Learning Models.'), ('/34352003/', 'A two level learning model for authorship authentication.'), ('/39226003/', 'A Holistic, Multi-Level, and Integrative Ethical Approach to Developing Machine Learning-Driven Decision Aids.'), ('/39283394/', 'Designing Regulatory Frameworks for Machine Learning Systems in Medicine-Time for Balance and Practicality.'), ('/39176811/', 'Benchmarking Approaches: Time Series Versus Feature-Based Machine Learning in ECG Analysis on the PTB-XL Dataset.')]
[('/38011820/', 'Reducing the vicissitudes of heterologous prochiral substrate catalysis by alcohol dehydrogenases through machine learning algorithms.'), ('/35931710/', 'A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification.'), ('/31470245/', "Predicting women's height from their socioeconomic status: A machine learning approach."), ('/34520937/', 'Biologically motivated learning method for deep neural networks using hierarchical competitive learning.'), ('/35773900/', 'Multinomial Classification of Neurosurgical Operations Using Gradient Boosting and Deep Learning Algorithms.'), ('/39225998/', 'Transparency, Evaluation and Going From "Ethics-Washing" to Enforceable Regulation: On Machine Learning-Driven Clinician Decision Aids.'), ('/38532161/', 'How AI is improving climate forecasts.'), ('/38253763/', 'Seven technologies to watch in 2024.'), ('/32716337/', 'Automatic classification of regular and irregular capnogram segments using time- and frequency-domain features: A machine learning-based approach.'), ('/38918522/', 'AI machine translation tools must be taught cultural differences too.')]
[('/31607596/', 'Looking to the future: Learning from experience, averting catastrophe.'), ('/32778970/', 'Estimation and easy calculation of the Palmer Drought Severity Index from the meteorological data by using the advanced machine learning algorithms.'), ('/29037046/', 'ADMET Evaluation in Drug Discovery. 18. Reliable Prediction of Chemical-Induced Urinary Tract Toxicity by Boosting Machine Learning Approaches.'), ('/29679305/', 'The New Possibilities from "Big Data" to Overlooked Associations Between Diabetes, Biochemical Parameters, Glucose Control, and Osteoporosis.'), ('/38159310/', 'Effluent parameters prediction of a biological nutrient removal (BNR) process using different machine learning methods: A case study.'), ('/35251155/', 'Application of Unsupervised Migration Method Based on Deep Learning Model in Basketball Training.'), ('/30422287/', 'On Deep Learning for Medical Image Analysis.'), ('/29799680/', 'MORE INTELLIGENCE FOR RADIOLOGY Machine learning offers promise, but much work lies ahead.'), ('/36100851/', 'An investigation of privacy preservation in deep learning-based eye-tracking.'), ('/37015653/', 'FedMix: Mixed Supervised Federated Learning for Medical Image Segmentation.')]
[('/33611065/', 'A survey on modern trainable activation functions.'), ('/38636345/', 'Thermogravimetric experiments based prediction of biomass pyrolysis behavior: A comparison of typical machine learning regression models in Scikit-learn.'), ('/34517520/', 'Malware detection based on semi-supervised learning with malware visualization.'), ('/38537150/', 'Prediction of Occult Hemorrhage in the Lower Body Negative Pressure Model: Initial Validation of Machine Learning Approaches.'), ('/32230844/', 'Evaluating the Impact of a Two-Stage Multivariate Data Cleansing Approach to Improve to the Performance of Machine Learning Classifiers: A Case Study in Human Activity Recognition.'), ('/35746191/', 'DOC-IDS: A Deep Learning-Based Method for Feature Extraction and Anomaly Detection in Network Traffic.'), ('/35437438/', 'The Modeling of Super Deep Learning Aiming at Knowledge Acquisition in Automatic Driving.'), ('/33924940/', 'Analysis of the Nosema Cells Identification for Microscopic Images.'), ('/36905822/', 'NLS: An accurate and yet easy-to-interpret prediction method.'), ('/39277634/', 'Bayesian optimized multimodal deep hybrid learning approach for tomato leaf disease classification.')]
[('/35393777/', 'DeepBBBP: High Accuracy Blood-brain-barrier Permeability Prediction with a Mixed Deep Learning Model.'), ('/37875622/', 'An AI revolution is brewing in medicine. What will it look like?'), ('/28427588/', 'Deep Learning With Unsupervised Feature in Echocardiographic Imaging.'), ('/35328048/', 'Comparative Study of Classification Algorithms for Various DNA Microarray Data.'), ('/36072742/', 'Optimization of Tennis Teaching Resources and Data Visualization Based on Support Vector Machine.'), ('/34956342/', 'Food Image Recognition and Food Safety Detection Method Based on Deep Learning.'), ('/29136004/', 'Predictability of machine learning techniques to forecast the trends of market index prices: Hypothesis testing for the Korean stock markets.'), ('/36109605/', 'Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning.'), ('/31147648/', 'Neuron segmentation with deep learning.'), ('/38874445/', 'Advancing Peptide-Based Cancer Therapy with AI: In-Depth Analysis of State-of-the-Art AI Models.')]
[('/34245871/', 'Extraction and assessment of diagnosis-relevant features for heart murmur classification.'), ('/29052645/', 'The future of work.'), ('/39158659/', 'Using algorithmic game theory to improve supervised machine learning: A novel applicability approach in flood susceptibility mapping.'), ('/31295300/', 'Ensemble-based kernel learning for a class of data assimilation problems with imperfect forward simulators.'), ('/35178081/', 'Vehicle Image Detection Method Using Deep Learning in UAV Video.'), ('/31320024/', 'Impact of Artificial Intelligence on Interventional Cardiology: From Decision-Making Aid to Advanced Interventional Procedure Assistance.'), ('/38607924/', 'Protocol to explain support vector machine predictions via exact Shapley value computation.'), ('/34862527/', 'Foundations of Feature Selection in Clinical Prediction Modeling.'), ('/28800619/', '"What is relevant in a text document?": An interpretable machine learning approach.'), ('/35655511/', 'Detection of Peripheral Malarial Parasites in Blood Smears Using Deep Learning Models.')]
[('/34300416/', 'A Comparative Study of Traffic Classification Techniques for Smart City Networks.'), ('/31437906/', 'An Ensemble Deep Learning Model for Drug Abuse Detection in Sparse Twitter-Sphere.'), ('/31438249/', 'Early Nephrosis Detection Based on Deep Learning with Clinical Time-Series Data.'), ('/35537509/', 'Compare the performance of multiple binary classification models in microbial high-throughput sequencing datasets.'), ('/38227587/', 'Geometric Deep Learning sub-network extraction for Maximum Clique Enumeration.'), ('/31147636/', 'Deep learning adds an extra dimension to peptide fragmentation.'), ('/36592526/', 'On joint parameterizations of linear and nonlinear functionals in neural networks.'), ('/38217632/', 'Multimodal feature fusion in deep learning for comprehensive dental condition classification.'), ('/37420549/', 'Combined CNN and RNN Neural Networks for GPR Detection of Railway Subgrade Diseases.'), ('/33065493/', 'Biomedical image classification made easier thanks to transfer and semi-supervised learning.')]
[('/38086938/', 'Can AI deliver advice that is judgement-free for science policy?'), ('/33118351/', 'Deep Learning Total Energies and Orbital Energies of Large Organic Molecules Using Hybridization of Molecular Fingerprints.'), ('/35868346/', 'An automatic method to quantify trichomes in Arabidopsis thaliana.'), ('/35674984/', 'Identifying molecular functional groups of organic compounds by deep learning of NMR data.'), ('/33826151/', 'A through-focus scanning optical microscopy dimensional measurement method based on deep-learning classification model.'), ('/36080936/', 'A Novel Prediction Model for Malicious Users Detection and Spectrum Sensing Based on Stacking and Deep Learning.'), ('/36248934/', 'Trends in Intelligent and AI-Based Software Engineering Processes: A Deep Learning-Based Software Process Model Recommendation Method.'), ('/30599419/', 'Research on a learning rate with energy index in deep learning.'), ('/29486381/', 'Manifold regularized matrix completion for multi-label learning with ADMM.'), ('/32361547/', 'Vulnerability of classifiers to evolutionary generated adversarial examples.')]
[('/32858170/', 'Sharpening the resolution on data matters: a brief roadmap for understanding deep learning for medical data.'), ('/33892544/', 'Inter classifier comparison to detect voice pathologies.'), ('/36274527/', 'Boundary heat diffusion classifier for a semi-supervised learning in a multilayer network embedding.'), ('/39283385/', 'The Fine Balance Between Complete Data Integrity in Medical Adaptive Machine Learning Systems and the Protection of Research Participants.'), ('/33544685/', 'Cross-Lingual Knowledge Transferring by Structural Correspondence and Space Transfer.'), ('/38052608/', 'Bird sound recognition based on adaptive frequency cepstral coefficient and improved support vector machine using a hunter-prey optimizer.'), ('/28072555/', 'On Constructing Ensembles for Combinatorial Optimisation.'), ('/32634104/', 'An Improved Performance of Deep Learning Based on Convolution Neural Network to Classify the Hand Motion by Evaluating Hyper Parameter.'), ('/37256849/', 'The effect of seasonality in predicting the level of crime. A spatial perspective.'), ('/35361114/', 'LANDMark: an ensemble approach to the supervised selection of biomarkers in high-throughput sequencing data.')]
[('/36256021/', 'Speckle classification of a multimode fiber based on Inception V3.'), ('/33171977/', 'Motion Inference Using Sparse Inertial Sensors, Self-Supervised Learning, and a New Dataset of Unscripted Human Motion.'), ('/31671043/', 'A Seventh Sense: Sentience and Surgical Robotics.'), ('/38308840/', 'Protocol to train a support vector machine for the automatic curation of bacterial cell detections in microscopy images.'), ('/29218875/', 'Mapping Patient Trajectories using Longitudinal Extraction and Deep Learning in the MIMIC-III Critical Care Database.'), ('/34960519/', 'Enhancing the Tracking of Seedling Growth Using RGB-Depth Fusion and Deep Learning.'), ('/32534378/', 'Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks.'), ('/30273844/', 'LCD: A Fast Contrastive Divergence Based Algorithm for Restricted Boltzmann Machine.'), ('/34058979/', 'A sequence-based multiple kernel model for identifying DNA-binding proteins.'), ('/32049525/', 'libmolgrid: Graphics Processing Unit Accelerated Molecular Gridding for Deep Learning Applications.')]
[('/39125808/', 'Developing a Semi-Supervised Approach Using a PU-Learning-Based Data Augmentation Strategy for Multitarget Drug Discovery.'), ('/35091229/', 'Transfer inhibitory potency prediction to binary classification: A model only needs a small training set.'), ('/35149486/', 'Graph convolutional neural network applied to the prediction of normal boiling point.'), ('/35402106/', 'Combining biomedical knowledge graphs and text to improve predictions for drug-target interactions and drug-indications.'), ('/38251798/', 'Image Recognition and Parameter Analysis of Concrete Vibration State Based on Support Vector Machine.'), ('/31445256/', 'Categorization of free-text drug orders using character-level recurrent neural networks.'), ('/31500931/', 'Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to Atari Breakout game.'), ('/35405216/', 'Prediction of product yields using fusion model from Co-pyrolysis of biomass and coal.'), ('/36463797/', 'Differentiating dreaming and waking reports with automatic text analysis and Support Vector Machines.'), ('/36365885/', 'A Novel Data Augmentation Method for Improving the Accuracy of Insulator Health Diagnosis.')]
[('/38877097/', 'Development of a real-time cattle lameness detection system using a single side-view camera.'), ('/30832569/', 'Architectures and accuracy of artificial neural network for disease classification from omics data.'), ('/33019275/', 'Novel Feature Selection for Artificial Intelligence Using Item Response Theory for Mortality Prediction.'), ('/37842747/', 'Classification of oolong tea varieties based on computer vision and convolutional neural networks.'), ('/39046926/', 'Developing a transit desert interactive dashboard: Supervised modeling for forecasting transit deserts.'), ('/38755269/', 'AI-based disease category prediction model using symptoms from low-resource Ethiopian language: Afaan Oromo text.'), ('/38622298/', 'AI now beats humans at basic tasks - new benchmarks are needed, says major report.'), ('/35161959/', 'Mixed Fault Classification of Sensorless PMSM Drive in Dynamic Operations Based on External Stray Flux Sensors.'), ('/31438159/', 'Skin Lesion Detection with Support Vector Machines on iOS Devices.'), ('/27708329/', 'Can we open the black box of AI?')]
[('/34451034/', 'Use of Hyperspectral Imaging for the Quantification of Organic Contaminants on Copper Surfaces for Electronic Applications.'), ('/39160848/', 'Development of a Predictive Model to Determine Appropriate Length of Profile.'), ('/33422928/', 'An enhanced approach to the robust discriminant analysis and class sparsity based embedding.'), ('/39053375/', 'Exploring the spatial patterns of landslide susceptibility assessment using interpretable Shapley method: Mechanisms of landslide formation in the Sichuan-Tibet region.'), ('/29925973/', 'Bias detectives: the researchers striving to make algorithms fair.'), ('/29684036/', 'Malay sentiment analysis based on combined classification approaches and Senti-lexicon algorithm.'), ('/33223272/', 'Characterization of specific and distinct patient types in clinical trials of acute schizophrenia using an uncorrelated PANSS score matrix transform (UPSM).'), ('/28358850/', 'Adaptive feature selection using v-shaped binary particle swarm optimization.'), ('/33866303/', 'Reducing bias to source samples for unsupervised domain adaptation.'), ('/33156855/', 'Evaluation of classification and forecasting methods on time series gene expression data.')]
[('/33018559/', 'Schrödinger Spectrum Based PPG Features for the Estimation of the Arterial Blood Pressure.'), ('/28238168/', 'Learning representation hierarchies by sharing visual features: a computational investigation of Persian character recognition with unsupervised deep learning.'), ('/26406166/', 'Extending XCS with Cyclic Graphs for Scalability on Complex Boolean Problems.'), ('/38641616/', 'Noisecut: a python package for noise-tolerant classification of binary data using prior knowledge integration and max-cut solutions.'), ('/34327638/', 'Developing a boosted decision tree regression prediction model as a sustainable tool for compressive strength of environmentally friendly concrete.'), ('/33158632/', 'Convolution neural network for effective burn region segmentation of color images.'), ('/32370162/', 'Method for Training Convolutional Neural Networks for In Situ Plankton Image Recognition and Classification Based on the Mechanisms of the Human Eye.'), ('/39160852/', 'Engineering Features From Advanced Medical Technology Initiative Submissions to Enable Predictive Modeling for Proposal Success.'), ('/30357558/', 'Massive Technological Unemployment Without Redistribution: A Case for Cautious Optimism.'), ('/31295691/', 'Depth with nonlinearity creates no bad local minima in ResNets.')]

多进程multiprocessing¶

  • 如果遇到了CPU密集型计算,多线程反而会降低执行速度
  • multiprocessing模块解决了GIL缺陷,用多进程在多CPU上并行执行

multiprocessing和threading对比¶

语法条目 多线程 多进程
引入模块 from threading import Thread from multiprocessing import Process
新建 t=Thread(target=func, args=(100,)) p=Process(target=f, args=('bob',))
启动 t.start() p.start()
等待结束 t.join() p.join()
通信 import queue
q=queue.Queue()
q.put(item)
item=q.get()
from multiprocessing import Queue
q=Queue()
q.put([42, None, 'hello'])
item=q.get()
语法条目 多线程 多进程
安全加锁 from threading import Lock
lock=Lock()
with lock:
# do somthing
from multiprocessing import Lock
lock=Lock()
with lock:
# do something
池化技术 from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor() as executor:
results=executor.map(func, [1,2,3])
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor() as executor:
results=executor.map(func, [1,2,3])

对于CPU密集任务,单线程,多线程和多进程对比¶

In [13]:
import math
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import ProcessPoolExecutor

def is_prime(number):
    if number < 2:
        return False
    if number == 2:
        return True
    else:
        ulim = math.ceil(math.sqrt(number))
        for k in range(3, ulim + 1, 2):
            if number % k == 0:
                return False
        return True
In [14]:
import random

def generate_random_numbers(count):
    numbers = []
    for i in range(count):
        number = random.randint(10**15, (10**16) - 1)
        numbers.append(number)
    return numbers

PRIMES = generate_random_numbers(200)
In [17]:
def single_thread():
    for number in PRIMES:
        is_prime(number)

def multi_thread():
    with ThreadPoolExecutor() as executor:
        results = executor.map(is_prime, PRIMES)

def multi_process():
    with ProcessPoolExecutor() as pool:
        results = pool.map(is_prime, PRIMES)

start = time.time()
multi_thread()
end = time.time()
print("multi thread cost", end - start)
start = time.time()
single_thread()
end = time.time()
print("single thread cost", end - start)
start = time.time()
multi_process()
end = time.time()
print("multi process cost", end - start)
multi thread cost 16.63011908531189
single thread cost 16.05925965309143
multi process cost 4.866028547286987
  • 由于GIL的存在,多线程比单线程计算的还慢,而多进程可以明显加快执行速度