Difference between revisions of "Machine Learning Experiments"

From Software Studies
Jump to navigation Jump to search
 
(56 intermediate revisions by 2 users not shown)
Line 6: Line 6:
*[http://blog.otoro.net/2017/01/01/recurrent-neural-network-artist/ Recurrent Neural Network Tutorial for Artists - with handwriting generation demo in p5.js]
*[http://blog.otoro.net/2017/01/01/recurrent-neural-network-artist/ Recurrent Neural Network Tutorial for Artists - with handwriting generation demo in p5.js]
* [https://arxiv.org/pdf/1801.00631.pdf Deep Learning: A Critical Appraisal] by Gary Marcus
* [https://arxiv.org/pdf/1801.00631.pdf Deep Learning: A Critical Appraisal] by Gary Marcus
* Caramiaux, Baptiste and Tanaka, Atau. 2013. Machine Learning of Musical Gestures: Principles and Review. Proceedings of the International Conference on New Interfaces for Musical Expression (NIME), pp. 513-518. [[https://research.gold.ac.uk/14645/ Article]]


==Cultural matters with Machine Learning==
==Cultural matters with Machine Learning==
*[https://mitpress.mit.edu/books/artificial-unintelligence] Artificial Unintelligence: How Computers Misunderstand the World by Meredith Broussard
* Kate Crawford and Trevor Paglen, “[https://www.excavating.ai/ Excavating AI: The Politics of Training Sets for Machine Learning] (September 19, 2019) https://excavating.ai
*[https://spectrum.ieee.org/tag/AI+history The six series of Untold History of AI] by Oscar Schwartz
*Parisi, Luciana, "[https://www.e-flux.com/journal/85/155472/reprogramming-decisionism/ Reprogramming Decisionism]", e-flux , Oct, 2017
*Kate Crawford and Vladan Joler, “Anatomy of an AI System: The Amazon Echo As An Anatomical Map of Human Labor, Data and Planetary Resources,” AI Now Institute and Share Lab, (September 7, 2018) [https://anatomyof.ai https://anatomyof.ai]
*[https://www.youtube.com/watch?v=fMym_BKWQzk The Trouble with Bias] NIPS 2017 Keynote by Kate Crawford
*[https://mitpress.mit.edu/books/artificial-unintelligence Artificial Unintelligence: How Computers Misunderstand the World] by Meredith Broussard
*[https://jods.mitpress.mit.edu/pub/costanza-chock Design Justice, A.I., and Escape from the Matrix of Domination] (2018) by Sasha Costanza-Chock
*[https://jods.mitpress.mit.edu/pub/costanza-chock Design Justice, A.I., and Escape from the Matrix of Domination] (2018) by Sasha Costanza-Chock
*[http://www.aprja.net/entanglement-machine-learning-and-human-ethics-in-driver-less-car-crashes/ Entanglement: Machine learning and human ethics in driver-less car crashes (2017)] by Maya Indira Ganesh
*[http://www.aprja.net/entanglement-machine-learning-and-human-ethics-in-driver-less-car-crashes/ Entanglement: Machine learning and human ethics in driver-less car crashes (2017)] by Maya Indira Ganesh
Line 20: Line 26:
*Mackenzie Adrian, [https://mitpress.mit.edu/books/machine-learners Machine Learners:Archaeology of a Data Practice], MIT Press, 2017
*Mackenzie Adrian, [https://mitpress.mit.edu/books/machine-learners Machine Learners:Archaeology of a Data Practice], MIT Press, 2017
*[https://github.com/eltiffster/authorFunction The Author Function: Imitating Grant Allen with Queer Writing Machines](2017) by Tiffany Chan (with source code)
*[https://github.com/eltiffster/authorFunction The Author Function: Imitating Grant Allen with Queer Writing Machines](2017) by Tiffany Chan (with source code)
*Cox,  Geoff. [https://unthinking.photography/themes/machine-vision/ways-of-machine-seeing Ways of machine seeing].  Unthinking Photography,  2016.
*Cox,  Geoff, [https://unthinking.photography/themes/machine-vision/ways-of-machine-seeing Ways of machine seeing].  Unthinking Photography,  2016.
*Miyazaki,  Shintaro,  [http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/89 Take Back the Algorithms! A Media Theory of Commonistic Affordance], (face recognition as a case), 2019.


==Technical explanation on Machine Learning==
==Technical explanation on Machine Learning==
===RNN/LSTM focus===
===RNN/LSTM focus===
*[http://ml4a.github.io/classes/itp-F18/08/ The Neural Aesthetic @ ITP-NYU, Fall 2018 - Lecture 8: Recurrent neural networks] (2018) by Gene Kogan
*[https://arxiv.org/abs/1308.0850 Generating sequences with recurrent neural networks](2013) by Graves, Alex.
*[http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Understanding LSTM Networks] (2015) by Christopher Colah
*[http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Understanding LSTM Networks] (2015) by Christopher Colah
*[http://karpathy.github.io/2015/05/21/rnn-effectiveness/ The Unreasonable Effectiveness of Recurrent Neural Networks] (2015) by Andrej Karpathy
*[http://karpathy.github.io/2015/05/21/rnn-effectiveness/ The Unreasonable Effectiveness of Recurrent Neural Networks] (2015) by Andrej Karpathy
*[https://skillsmatter.com/skillscasts/6611-visualizing-and-understanding-recurrent-networks Visualizing and Understanding Recurrent Networks (Video)] (2015) by Andrej Karpathy
*[https://www.mitpressjournals.org/doi/pdf/10.1162/089976600300015015 Learning to Forget: Continual Prediction with LSTM] (2000) by Felix A. Gers, Jürgen Schmidhuber and Fred Cummins
*[https://www.youtube.com/watch?v=56TYLaQN4N8 Deep Learning Lecture 12: Recurrent Neural Nets and LSTMs (video)] (2015) by Nando de Freitas (start at 4:10)


===Neural Network===
===Neural Network===
*[https://medium.freecodecamp.org/how-to-create-a-neural-network-in-javascript-in-only-30-lines-of-code-343dafc50d49 How to create a Neural Network in JavaScript in only 30 lines of code] (2017) by Per Harald Borgen, see sceencast [https://scrimba.com/casts/cast-1980 here]. Source code [https://github.com/siusoon/ML_neuralnetworkcreation here]
*[https://medium.freecodecamp.org/how-to-create-a-neural-network-in-javascript-in-only-30-lines-of-code-343dafc50d49 How to create a Neural Network in JavaScript in only 30 lines of code] (2017) by Per Harald Borgen, see sceencast [https://scrimba.com/casts/cast-1980 here]. Source code [https://github.com/siusoon/ML_neuralnetworkcreation here]
*[https://eisenjulian.github.io/deep-learning-in-100-lines/ Neural Networks in 100 lines of pure Python] (2019) by Julian Eisenschlos
===Data Processing===
*[https://www.datacamp.com/community/tutorials/preprocessing-in-data-science-part-1-centering-scaling-and-knn Preprocessing in Data Science (Part 1): Centering, Scaling, and KNN] by Hugo Bowne-Anderson


==Examples with source code==
==Examples with source code==
===Mixed media (text/image)===
===Mixed media (text/image)===
* [https://ml5js.org/ ML5.js library] developed by NYU ITP, text training tutorial [https://blog.paperspace.com/training-an-lstm-and-using-the-model-in-ml5-js/ here]
* [https://ml5js.org/ ML5.js library] developed by NYU ITP, text training tutorial [https://blog.paperspace.com/training-an-lstm-and-using-the-model-in-ml5-js/ here]
* [https://scikit-learn.org/stable/ scikit-learn]: Machine Learning in Python


===Text related===
===Text related===
* [https://github.com/gsurma/text_predictor text predictor] by Greg Surma (python 2.7/3.+ with tensorflow (RNN+LSTM) and his article here: [https://towardsdatascience.com/text-predictor-generating-rap-lyrics-with-recurrent-neural-networks-lstms-c3a1acbbda79 Text Predictor - Generating Rap Lyrics with Recurrent Neural Networks (LSTMs)]
* [https://github.com/fukuball/Tom-Chang-Deep-Lyrics Tom-Chang-Deep-Lyrics | 基於 LSTM 深度學習方法研發而成](2018) by 林志傑 (Python with bleach-1.5.0 html5lib-0.9999999 tensorboard-1.8.0 tensorflow-1.8.0)
* [https://colab.research.google.com/drive/1XlmtcyMdPRQC6bw2HQYb3UPtVGKqUJ0a#scrollTo=8R0T0ei52FXS Semantic similarity chatbot (with movie dialog)] by Allison Parrish
* [https://colab.research.google.com/drive/1XlmtcyMdPRQC6bw2HQYb3UPtVGKqUJ0a#scrollTo=8R0T0ei52FXS Semantic similarity chatbot (with movie dialog)] by Allison Parrish
* Chinese receipt OCR using Tensorflow | SpikeFlow ([https://deeperic.wordpress.com/2017/02/18/chinese-ocr-tensorflow/ Blog] | [https://github.com/deeperic/SpikeFlow Github])
* Chinese receipt OCR using Tensorflow | SpikeFlow ([https://deeperic.wordpress.com/2017/02/18/chinese-ocr-tensorflow/ Blog] | [https://github.com/deeperic/SpikeFlow Github])
* [https://github.com/karpathy/recurrentjs Recurrentjs] by Andrej Karpathy, mainly for text training. "Sentences are input data and the networks are trained to predict the next character in a sentence." + his [https://www.datascienceweekly.org/data-scientist-interviews/training-deep-learning-models-browser-andrej-karpathy-interview interview] on why javascript and machine learning.
* [https://github.com/karpathy/recurrentjs Recurrentjs] by Andrej Karpathy, mainly for text training. "Sentences are input data and the networks are trained to predict the next character in a sentence." + his [https://www.datascienceweekly.org/data-scientist-interviews/training-deep-learning-models-browser-andrej-karpathy-interview interview] on why javascript and machine learning.
* Re-appropriation of Recurrentjs by [http://artist.ai/index.php/home/poems/ UCL Creative Hub]
* Re-appropriation of Recurrentjs by [http://artist.ai/index.php/home/poems/ UCL Creative Hub]
===Chinese text related===
* [https://www.jiqizhixin.com/articles/2019-03-21-9 达观数据:综述中英文自然语言处理的异和同] (2019) by 机器之心
===Image related===
===Image related===
* [TensorFlow.js Coco SSD's model object detection](https://github.com/juandes/tensorflowjs-objectdetection-tutorial)by Juan De Dios Santos
* Image paint: 1/ [http://cs.stanford.edu/people/karpathy/convnetjs/ ConvNetJS Library] by Andrej Karpathy 2/ [http://caza.la/synaptic/#/paint-an-image Synaptic.js] by Synaptic
* Image paint: 1/ [http://cs.stanford.edu/people/karpathy/convnetjs/ ConvNetJS Library] by Andrej Karpathy 2/ [http://caza.la/synaptic/#/paint-an-image Synaptic.js] by Synaptic
* [https://affinelayer.com/pixsrv/index.html Image-to-Image Demo: Interactive Image Translation with pix2pix-tensorflow] by Christopher Hesse
===Sound related===
===Sound related===
* [http://prostheticknowledge.tumblr.com/post/170013157626/deep-roll-project-by-tero-parviainen-generates Deep roll] by Tero Parviainen | [https://codepen.io/teropa/full/zpbLOj/ source code] built with TensorFlow and js
* [http://prostheticknowledge.tumblr.com/post/170013157626/deep-roll-project-by-tero-parviainen-generates Deep roll] by Tero Parviainen | [https://codepen.io/teropa/full/zpbLOj/ source code] built with TensorFlow and js
Line 50: Line 76:
*[https://medium.com/@samim/obama-rnn-machine-generated-political-speeches-c8abd18a2ea0 Obama RNN] by Samim
*[https://medium.com/@samim/obama-rnn-machine-generated-political-speeches-c8abd18a2ea0 Obama RNN] by Samim
*[https://twitter.com/RNN_Bible RNN Bible on Twitter]
*[https://twitter.com/RNN_Bible RNN Bible on Twitter]
 
*[https://chatbotslife.com/notes-on-remixing-noon-generative-text-and-markov-chains-84ff4ec23937 Notes on remixing Noon, generative text and Markov chains] by Rev Dan Catt
===Artworks===
===Artworks===
*[http://runme.org/categories/+artificial_intelligence/ AI projects in software art repository/runme.org]
*[http://concept-script.com/Errant/index.html Errant: The Kinetic Propensity of Images] by Hector Rodriguez (2018)
*[https://andreasrefsgaard.dk/project/an-algorithm-watching-a-movie-trailer/ An algorithm watching a movie trailer] by Andreas Refsgaard and Lasse Korsgaard
*[https://sofianaudry.com/en/of-the-soone of the soone] by Sofian Audry (2017)
*[http://www.helenpritchard.info/critter-compiler-prototype-2016 Critter Compiler] prototype by Helen Pritchard (2016)
*[http://www.helenpritchard.info/critter-compiler-prototype-2016 Critter Compiler] prototype by Helen Pritchard (2016)
*[http://cloudindx.com/ Cloud index] by James Bridle | Read his interview [https://medium.com/intersections-arts-and-digital-culture-in-the-uk/james-bridle-machine-learning-in-practice-d7cb58cd20cb here on James Bridle: Machine Learning in Practice]
*[http://cloudindx.com/ Cloud index] by James Bridle | Read his interview [https://medium.com/intersections-arts-and-digital-culture-in-the-uk/james-bridle-machine-learning-in-practice-d7cb58cd20cb here on James Bridle: Machine Learning in Practice]
Line 66: Line 96:


===Workshop===
===Workshop===
*[https://anatomiesofintelligence.github.io/workshop_presentation.html Anatomies of Intelligence] (2019), led by Joana Chicau and Jonathan Reus 
*[https://www.vam.ac.uk/blog/museum-life/open-call-the-work-of-art-in-the-age-of-artificial-intelligence-workshop The Work of Art in the Age of Artificial Intelligence Workshop] (2018) by media art curator Natalia Fuchs and media artist Helena Nikonole
*[https://www.vam.ac.uk/blog/museum-life/open-call-the-work-of-art-in-the-age-of-artificial-intelligence-workshop The Work of Art in the Age of Artificial Intelligence Workshop] (2018) by media art curator Natalia Fuchs and media artist Helena Nikonole
*[http://www.spacestudios.org.uk/art-technology/counting-to-4-0-1-2-3-data-collection-as-art-practice-protest/ Counting to 4: 0, 1, 2, 3 – Data Collection as Art Practice & Protest] (2017) by Caroline Sinders
*[http://www.spacestudios.org.uk/art-technology/counting-to-4-0-1-2-3-data-collection-as-art-practice-protest/ Counting to 4: 0, 1, 2, 3 – Data Collection as Art Practice & Protest] (2017) by Caroline Sinders


===Conference===
===Conference===
*[http://aias.au.dk/events/aiasconference-musicandartificialintelligence/ Music and Artificial Intelligence - Pasts and Futures, Opportunities and Risks] (2019) organized by Georgina Born, AIAS, Aarhus University
*[https://www.cityu.edu.hk/iscma/ Art Machines: International Symposium on Computational Media Art] (2019) organized by School of Creative Media, City University of Hong Kong
*[https://www.cityu.edu.hk/iscma/ Art Machines: International Symposium on Computational Media Art] (2019) organized by School of Creative Media, City University of Hong Kong
*[https://zkm.de/en/event/2018/04/encoding-cultures-living-amongst-intelligent-machines international conference »Encoding Cultures. Living Amongst Intelligent Machines«](2018)
*[https://zkm.de/en/event/2018/04/encoding-cultures-living-amongst-intelligent-machines international conference »Encoding Cultures. Living Amongst Intelligent Machines«](2018)
Line 80: Line 112:
*[https://storage.googleapis.com/tfjs-examples/webcam-transfer-learning/dist/index.html Webcam Pacman] by BANDAI NAMCO Entertainment Inc.
*[https://storage.googleapis.com/tfjs-examples/webcam-transfer-learning/dist/index.html Webcam Pacman] by BANDAI NAMCO Entertainment Inc.
*[https://andreasref.github.io/p5js_knn_demos/ P5js KNN demos] by Andreas Refsgaard
*[https://andreasref.github.io/p5js_knn_demos/ P5js KNN demos] by Andreas Refsgaard
*[https://github.com/tensorflow/tfjs/blob/master/GALLERY.md TensorFlow.js gallery]


==Learning resource==
==Learning resource==
Line 92: Line 125:
* [http://cs231n.stanford.edu/syllabus.html CS231n: Convolutional Neural Networks for Visual Recognition] by Standford
* [http://cs231n.stanford.edu/syllabus.html CS231n: Convolutional Neural Networks for Visual Recognition] by Standford
* [http://sharedli.st/jhave2 Machine Learning resource list] by David Jhave Johnston
* [http://sharedli.st/jhave2 Machine Learning resource list] by David Jhave Johnston
* [https://www.youtube.com/watch?v=D7ZL45xS39I&feature=youtu.be Machine Learning Magic for Your Javascript Application (Google I/O'19)] with Tensorflow.js [Video] (2019)
* [https://www.tensorflow.org/alpha/tutorials/generative/dcgan Deep Convolutional Generative Adversarial Network] Step by step with python code walkthrough


==Teaching Machine Learning==
==Teaching Machine Learning==
Line 97: Line 132:


==Experiments/Tests==
==Experiments/Tests==
*01/2020: TensorFlow.js + React, object detection
[[File:objectDetection.png|none|300px|object]]
*08/2019: Runway's image to text prediction: "a collage of photos of a man on a motorcycle"
[[File:Runway3.png|none|300px|runway3]]
*08/2019: Runway's YOLACT - computer and algorithm shift our attention to something else. A lid cannot be recognized but that's the key object in the photo (ways of human seeing) actually. May be the training data contains mostly western objects. [[File:Runway1.png|none|300px|runway1]] [[File:Runway2.png|none|300px|runway2]]
*08/2019: Running LSTM/tensorflow with text_predictor.py and ml5 training
[[File:Predictor.png|none|400px|training both]]
*12/2018: Try running LSTM/tensorflow training on chinese text with python (again following text predictor)
[[File:Screen_Shot_2018-12-09_at_23.40.09.png|none|400px|weiboscope]]
*11/2018: Try running LSTM/tensorflow training with Python (following text predictor)from my PhD thesis text
[[File:Text predictor.png|none|400px|PhD thesis]]
*11/2018: Try running local ml5 + python training with English text (multiple manifestos) and generate 10000 characters text from multiple manifestos
[[File:Training.png|none|400px|manifestos]]
*06/2018: Try running LSTM ml5js with training simplied chinese data. Source from weiboscope 2012 week 1 deleted text
*06/2018: Try running LSTM ml5js with training simplied chinese data. Source from weiboscope 2012 week 1 deleted text
[[File:erasedText.jpg|none|400px|weiboscope text]]
[[File:erasedText.jpg|none|400px|weiboscope text]]

Latest revision as of 17:35, 12 January 2020

Introduction to Machine Learning

Cultural matters with Machine Learning

Technical explanation on Machine Learning

RNN/LSTM focus

Neural Network

Data Processing

Examples with source code

Mixed media (text/image)

Text related

Chinese text related

Image related

Sound related

Projects

Text related

Artworks

Examples/Performance/Speculative design

Exhibition

Workshop

Conference

Preconference Workshop] (2018)

Demo/Experimental Projects

Learning resource

Teaching Machine Learning

Experiments/Tests

  • 01/2020: TensorFlow.js + React, object detection
object
  • 08/2019: Runway's image to text prediction: "a collage of photos of a man on a motorcycle"
runway3
  • 08/2019: Runway's YOLACT - computer and algorithm shift our attention to something else. A lid cannot be recognized but that's the key object in the photo (ways of human seeing) actually. May be the training data contains mostly western objects.
    runway1
    runway2
  • 08/2019: Running LSTM/tensorflow with text_predictor.py and ml5 training
training both
  • 12/2018: Try running LSTM/tensorflow training on chinese text with python (again following text predictor)
weiboscope
  • 11/2018: Try running LSTM/tensorflow training with Python (following text predictor)from my PhD thesis text
PhD thesis
  • 11/2018: Try running local ml5 + python training with English text (multiple manifestos) and generate 10000 characters text from multiple manifestos
manifestos
  • 06/2018: Try running LSTM ml5js with training simplied chinese data. Source from weiboscope 2012 week 1 deleted text
weiboscope text
  • 06/2018: Running LSTM ml5js example with my own training data
Training Process
Outcome Process
  • 2018: Running ml5.js example - Simple LSTM Generator Example on a local browser
Predicting what's the text
  • Running ml5.js example - Simple Image Classification Example on a local browser
Predicting what's the image with confidence level
  • Running spam data with RecurrentJS on a local browser
Running spam data with RecurrentJS
  • Running a customized neural network on a local browser
Learning XOR with Synaptic
  • Running a PNG file with Synaptic.js on a local browser
Learning a png file with Synaptic
  • Running a jpg file with ConvNetJS on a local browser
Learning a jpg file with ConvNetJS