Difference between revisions of "Machine Learning Experiments"

From Software Studies
Jump to navigation Jump to search
 
(185 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==References==
==Introduction to Machine Learning==
*[https://vimeo.com/273813642 Machine Learning for Creative Media] [Video] 10 series workshop by Gene Kogan
*[https://www.youtube.com/watch?v=POrPIABj2MI A Return to Machine Learning] [Video] by Kyle McDonald
*[https://www.youtube.com/watch?v=nKW8Ndu7Mjw&index=2&list=PLIivdWyY5sqJxnwJhe3etaK7utrBiPBQ2 The 7 steps of Machine Learning] (2017) [Video] by Yufeng/Google Cloud
*[http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ A visual introduction to machine learning] by r2d3.
*[http://blog.otoro.net/2017/01/01/recurrent-neural-network-artist/ Recurrent Neural Network Tutorial for Artists - with handwriting generation demo in p5.js]
* [https://arxiv.org/pdf/1801.00631.pdf Deep Learning: A Critical Appraisal] by Gary Marcus
* Caramiaux, Baptiste and Tanaka, Atau. 2013. Machine Learning of Musical Gestures: Principles and Review. Proceedings of the International Conference on New Interfaces for Musical Expression (NIME), pp. 513-518. [[https://research.gold.ac.uk/14645/ Article]]


==Examples==
==Cultural matters with Machine Learning==
* Kate Crawford and Trevor Paglen, “[https://www.excavating.ai/ Excavating AI: The Politics of Training Sets for Machine Learning] (September 19, 2019) https://excavating.ai
*[https://spectrum.ieee.org/tag/AI+history The six series of Untold History of AI] by Oscar Schwartz
*Parisi, Luciana, "[https://www.e-flux.com/journal/85/155472/reprogramming-decisionism/ Reprogramming Decisionism]", e-flux , Oct, 2017
*Kate Crawford and Vladan Joler, “Anatomy of an AI System: The Amazon Echo As An Anatomical Map of Human Labor, Data and Planetary Resources,” AI Now Institute and Share Lab, (September 7, 2018) [https://anatomyof.ai https://anatomyof.ai]
*[https://www.youtube.com/watch?v=fMym_BKWQzk The Trouble with Bias] NIPS 2017 Keynote by Kate Crawford
*[https://mitpress.mit.edu/books/artificial-unintelligence Artificial Unintelligence: How Computers Misunderstand the World] by Meredith Broussard
*[https://jods.mitpress.mit.edu/pub/costanza-chock Design Justice, A.I., and Escape from the Matrix of Domination] (2018) by Sasha Costanza-Chock
*[http://www.aprja.net/entanglement-machine-learning-and-human-ethics-in-driver-less-car-crashes/ Entanglement: Machine learning and human ethics in driver-less car crashes (2017)] by Maya Indira Ganesh
*[https://docs.google.com/document/d/11dVbSJblYHSgD1kUqKwrAPm0C7dOPs7vd9ywxsXYylU/edit Machine Learning and the Complexities of Human Emotions] by Caroline Sinders
*[http://www.bath.ac.uk/research/news/2017/04/13/biased-bots-artificial-intelligence/ Biased bots: Human prejudices sneak into AI systems] by Joanna Bryson 
*[https://quamproxime.com/2017/07/30/the-temporality-of-artificial-intelligence/ The Temporality of Artificial Intelligence (2017)] by Kathryn Hume
*Mackenzie Adrian, [http://www.realtechsupport.org/UB/ML+CT/papers/Mackenzie_Production_of_Prediction_2015.pdf The production of prediction: What does  machine learning want?],European Journal of Cultural Studies2015, Vol.18(4-5) 429–445
*[https://github.com/rian39/warwick_one/blob/master/warwick_presentation.md In the data: interdisciplinary modes of machine learning] by Adrian Mackenzie
*[https://www.academia.edu/34471806/Automating_Aesthetics_Artificial_Intelligence_and_Image_Culture?auto=download&campaign=weekly_digest Automating Aesthetics: Artificial Intelligence and Image Culture] (2017) by Lev Manovich
*Ed Finn, "Building the Star Trek Computer," in [https://mitpress.mit.edu/books/what-algorithms-want What Algorithms Want], MIT Press, 2017, pp. 57-85.
*Mackenzie Adrian, [https://mitpress.mit.edu/books/machine-learners Machine Learners:Archaeology of a Data Practice], MIT Press, 2017
*[https://github.com/eltiffster/authorFunction The Author Function: Imitating Grant Allen with Queer Writing Machines](2017) by Tiffany Chan (with source code)
*Cox,  Geoff,  [https://unthinking.photography/themes/machine-vision/ways-of-machine-seeing Ways of machine seeing].  Unthinking Photography,  2016.
*Miyazaki,  Shintaro,  [http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/89 Take Back the Algorithms! A Media Theory of Commonistic Affordance], (face recognition as a case), 2019.
 
==Technical explanation on Machine Learning==
===RNN/LSTM focus===
*[http://ml4a.github.io/classes/itp-F18/08/ The Neural Aesthetic @ ITP-NYU, Fall 2018 - Lecture 8: Recurrent neural networks] (2018) by Gene Kogan
*[https://arxiv.org/abs/1308.0850 Generating sequences with recurrent neural networks](2013) by Graves, Alex.
*[http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Understanding LSTM Networks] (2015) by Christopher Colah
*[http://karpathy.github.io/2015/05/21/rnn-effectiveness/ The Unreasonable Effectiveness of Recurrent Neural Networks] (2015) by Andrej Karpathy
*[https://skillsmatter.com/skillscasts/6611-visualizing-and-understanding-recurrent-networks Visualizing and Understanding Recurrent Networks (Video)] (2015) by Andrej Karpathy
*[https://www.mitpressjournals.org/doi/pdf/10.1162/089976600300015015 Learning to Forget: Continual Prediction with LSTM] (2000) by Felix A. Gers, Jürgen Schmidhuber and Fred Cummins
*[https://www.youtube.com/watch?v=56TYLaQN4N8 Deep Learning Lecture 12: Recurrent Neural Nets and LSTMs (video)] (2015) by Nando de Freitas (start at 4:10)
 
===Neural Network===
*[https://medium.freecodecamp.org/how-to-create-a-neural-network-in-javascript-in-only-30-lines-of-code-343dafc50d49 How to create a Neural Network in JavaScript in only 30 lines of code] (2017) by Per Harald Borgen, see sceencast [https://scrimba.com/casts/cast-1980 here]. Source code [https://github.com/siusoon/ML_neuralnetworkcreation here]
*[https://eisenjulian.github.io/deep-learning-in-100-lines/ Neural Networks in 100 lines of pure Python] (2019) by Julian Eisenschlos
 
===Data Processing===
*[https://www.datacamp.com/community/tutorials/preprocessing-in-data-science-part-1-centering-scaling-and-knn Preprocessing in Data Science (Part 1): Centering, Scaling, and KNN] by Hugo Bowne-Anderson
 
==Examples with source code==
===Mixed media (text/image)===
* [https://ml5js.org/ ML5.js library] developed by NYU ITP, text training tutorial [https://blog.paperspace.com/training-an-lstm-and-using-the-model-in-ml5-js/ here]
* [https://scikit-learn.org/stable/ scikit-learn]: Machine Learning in Python
 
===Text related===
* [https://github.com/gsurma/text_predictor text predictor] by Greg Surma (python 2.7/3.+ with tensorflow (RNN+LSTM) and his article here: [https://towardsdatascience.com/text-predictor-generating-rap-lyrics-with-recurrent-neural-networks-lstms-c3a1acbbda79 Text Predictor - Generating Rap Lyrics with Recurrent Neural Networks (LSTMs)]
* [https://github.com/fukuball/Tom-Chang-Deep-Lyrics Tom-Chang-Deep-Lyrics | 基於 LSTM 深度學習方法研發而成](2018) by 林志傑 (Python with bleach-1.5.0 html5lib-0.9999999 tensorboard-1.8.0 tensorflow-1.8.0)
* [https://colab.research.google.com/drive/1XlmtcyMdPRQC6bw2HQYb3UPtVGKqUJ0a#scrollTo=8R0T0ei52FXS Semantic similarity chatbot (with movie dialog)] by Allison Parrish
* Chinese receipt OCR using Tensorflow | SpikeFlow ([https://deeperic.wordpress.com/2017/02/18/chinese-ocr-tensorflow/ Blog] | [https://github.com/deeperic/SpikeFlow Github])
* Chinese receipt OCR using Tensorflow | SpikeFlow ([https://deeperic.wordpress.com/2017/02/18/chinese-ocr-tensorflow/ Blog] | [https://github.com/deeperic/SpikeFlow Github])
* [https://github.com/karpathy/recurrentjs Recurrentjs], mainly for text training. "Sentences are input data and the networks are trained to predict the next character in a sentence."
* [https://github.com/karpathy/recurrentjs Recurrentjs] by Andrej Karpathy, mainly for text training. "Sentences are input data and the networks are trained to predict the next character in a sentence." + his [https://www.datascienceweekly.org/data-scientist-interviews/training-deep-learning-models-browser-andrej-karpathy-interview interview] on why javascript and machine learning.
* Re-appropriation of Recurrentjs by [http://artist.ai/index.php/home/poems/ UCL Creative Hub]
 
===Chinese text related===
* [https://www.jiqizhixin.com/articles/2019-03-21-9 达观数据:综述中英文自然语言处理的异和同] (2019) by 机器之心
 
===Image related===
* [TensorFlow.js Coco SSD's model object detection](https://github.com/juandes/tensorflowjs-objectdetection-tutorial)by Juan De Dios Santos
* Image paint: 1/ [http://cs.stanford.edu/people/karpathy/convnetjs/ ConvNetJS Library] by Andrej Karpathy 2/ [http://caza.la/synaptic/#/paint-an-image Synaptic.js] by Synaptic
* [https://affinelayer.com/pixsrv/index.html Image-to-Image Demo: Interactive Image Translation with pix2pix-tensorflow] by Christopher Hesse
 
===Sound related===
* [http://prostheticknowledge.tumblr.com/post/170013157626/deep-roll-project-by-tero-parviainen-generates Deep roll] by Tero Parviainen | [https://codepen.io/teropa/full/zpbLOj/ source code] built with TensorFlow and js
 
==Projects==
===Text related===
*[http://www.cs.toronto.edu/~ilya/rnn.html THE TEXT-GENERATING RNN DEMO by Ilya Sutskever]
*[http://www.cs.toronto.edu/~graves/handwriting.html Handwriting Generation Demo by Alex Graves]
*[https://medium.com/@samim/obama-rnn-machine-generated-political-speeches-c8abd18a2ea0 Obama RNN] by Samim
*[https://twitter.com/RNN_Bible RNN Bible on Twitter]
*[https://chatbotslife.com/notes-on-remixing-noon-generative-text-and-markov-chains-84ff4ec23937 Notes on remixing Noon, generative text and Markov chains] by Rev Dan Catt
===Artworks===
*[http://runme.org/categories/+artificial_intelligence/ AI projects in software art repository/runme.org]
*[http://concept-script.com/Errant/index.html Errant: The Kinetic Propensity of Images] by Hector Rodriguez (2018)
*[https://andreasrefsgaard.dk/project/an-algorithm-watching-a-movie-trailer/ An algorithm watching a movie trailer] by Andreas Refsgaard and Lasse Korsgaard
*[https://sofianaudry.com/en/of-the-soone of the soone] by Sofian Audry (2017)
*[http://www.helenpritchard.info/critter-compiler-prototype-2016 Critter Compiler] prototype by Helen Pritchard (2016)
*[http://cloudindx.com/ Cloud index] by James Bridle | Read his interview [https://medium.com/intersections-arts-and-digital-culture-in-the-uk/james-bridle-machine-learning-in-practice-d7cb58cd20cb here on James Bridle: Machine Learning in Practice]
*[http://bdp.glia.ca/about/ Big Data Poetry] by David Jhave Johnston
*[http://www.memo.tv/learning-to-see-hello-world/ Learning to see: Hello, World!] (2017) by Memo Akten. More [http://www.memo.tv/category/work/by-type/ here]
 
===Examples/Performance/Speculative design===
*[http://www.wekinator.org/example-projects/ Examples of Wekinator]
*[https://vimeo.com/247788194 AYA pushes back] by Marie Louise Juul Søndergaard
 
===Exhibition===
*[http://www.architectmagazine.com/design/exhibits-books-etc/can-computers-think-creatively_o Artists & Robots (2017)] at the Astana Contemporary Art Center in Kazakhstan
*[https://www.aec.at/ai/en/theme/ Artificial Intelligence: The Other I (2017)] Ars Electronica
 
===Workshop===
*[https://anatomiesofintelligence.github.io/workshop_presentation.html Anatomies of Intelligence] (2019), led by Joana Chicau and Jonathan Reus 
*[https://www.vam.ac.uk/blog/museum-life/open-call-the-work-of-art-in-the-age-of-artificial-intelligence-workshop The Work of Art in the Age of Artificial Intelligence Workshop] (2018) by media art curator Natalia Fuchs and media artist Helena Nikonole
*[http://www.spacestudios.org.uk/art-technology/counting-to-4-0-1-2-3-data-collection-as-art-practice-protest/ Counting to 4: 0, 1, 2, 3 – Data Collection as Art Practice & Protest] (2017) by Caroline Sinders
 
===Conference===
*[http://aias.au.dk/events/aiasconference-musicandartificialintelligence/ Music and Artificial Intelligence - Pasts and Futures, Opportunities and Risks] (2019) organized by Georgina Born, AIAS, Aarhus University
*[https://www.cityu.edu.hk/iscma/ Art Machines: International Symposium on Computational Media Art] (2019) organized by School of Creative Media, City University of Hong Kong
*[https://zkm.de/en/event/2018/04/encoding-cultures-living-amongst-intelligent-machines international conference »Encoding Cultures. Living Amongst Intelligent Machines«](2018)
*[https://aoir.org/aoir2018/preconfwrkshop/#CL The Cultural Life of Machine Learning: An Incursion into Critical AI Studies
Preconference Workshop] (2018)
 
===Demo/Experimental Projects===
*[https://teachablemachine.withgoogle.com/ Teachable Machine], example [https://www.youtube.com/watch?v=oP8-_0ZyY3U&feature=youtu.be Rock out by wiggling your fingers], source code [https://github.com/googlecreativelab/teachable-machine here]
*[https://experiments.withgoogle.com/ai/giorgio-cam Giorgio Cam] by Eric Rosenbaum and Yotam Mann, source code [https://github.com/googlecreativelab/aiexperiments-giorgio-cam here]
*[https://storage.googleapis.com/tfjs-examples/webcam-transfer-learning/dist/index.html Webcam Pacman] by BANDAI NAMCO Entertainment Inc.
*[https://andreasref.github.io/p5js_knn_demos/ P5js KNN demos] by Andreas Refsgaard
*[https://github.com/tensorflow/tfjs/blob/master/GALLERY.md TensorFlow.js gallery]


==Learning resource==
==Learning resource==
* [https://itp.nyu.edu/adjacent/issue-3/ml5-friendly-open-source-machine-learning-library-for-the-web/ ml5: Friendly Open Source Machine Learning Library for the Web] by Daniel Shiffman and ml5.js collaborators
* Daniel Shiffman's Nature of Code: [http://natureofcode.com/book/chapter-10-neural-networks/ Neural networks]| [https://www.youtube.com/watch?v=XJ7HLz9VYz0&list=PLRqwX-V7Uu6aCibgK1PTWWu9by6XFdCfh video]
* Daniel Shiffman's Nature of Code: [http://natureofcode.com/book/chapter-10-neural-networks/ Neural networks]| [https://www.youtube.com/watch?v=XJ7HLz9VYz0&list=PLRqwX-V7Uu6aCibgK1PTWWu9by6XFdCfh video]
* [https://ml4a.github.io/ Machine Learning for Artists] by Gene Kogan and Francis Tseng
* [https://ml4a.github.io/ Machine Learning for Artists] by Gene Kogan and Francis Tseng
* [https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists/info Machine Learning for Muscians and Artists] by Rebecca Fiebrink (main), Laetitia Sonami (guest) and Baptiste Caramiaux (guest)
* [https://www.coursera.org/learn/machine-learning/ Machine Learning, taught by Andrew Ng]
* [https://www.udemy.com/machinelearning/ Machine Learning A-Z™: Hands-On Python & R In Data Science]
* [https://www.udemy.com/machinelearning/ Machine Learning A-Z™: Hands-On Python & R In Data Science]
* [http://blog.otoro.net/2016/05/07/backprop-neat/ Neural Network Evolution Playground with Backprop NEAT]
* [https://www.youtube.com/watch?v=nAHcrz5hxc4 ConvNetJS: Deep Learning in the Browser] [Video] by Christoph Körner (more technical)
* [http://cs231n.stanford.edu/syllabus.html CS231n: Convolutional Neural Networks for Visual Recognition] by Standford
* [http://sharedli.st/jhave2 Machine Learning resource list] by David Jhave Johnston
* [https://www.youtube.com/watch?v=D7ZL45xS39I&feature=youtu.be Machine Learning Magic for Your Javascript Application (Google I/O'19)] with Tensorflow.js [Video] (2019)
* [https://www.tensorflow.org/alpha/tutorials/generative/dcgan Deep Convolutional Generative Adversarial Network] Step by step with python code walkthrough
==Teaching Machine Learning==
*[http://alimomeni.net/machine-learning-for-introductory-physical-computing-curricula/ Machine Learning for Introductory Physical Computing Curricula] by Ali Momeni


==Experiments/Tests==
==Experiments/Tests==
[[File:Learningspam.png|100px|Running spam data in RecurrentJS]]
*01/2020: TensorFlow.js + React, object detection
[[File:objectDetection.png|none|300px|object]]
*08/2019: Runway's image to text prediction: "a collage of photos of a man on a motorcycle"
[[File:Runway3.png|none|300px|runway3]]
*08/2019: Runway's YOLACT - computer and algorithm shift our attention to something else. A lid cannot be recognized but that's the key object in the photo (ways of human seeing) actually. May be the training data contains mostly western objects. [[File:Runway1.png|none|300px|runway1]] [[File:Runway2.png|none|300px|runway2]]
*08/2019: Running LSTM/tensorflow with text_predictor.py and ml5 training
[[File:Predictor.png|none|400px|training both]]
*12/2018: Try running LSTM/tensorflow training on chinese text with python (again following text predictor)
[[File:Screen_Shot_2018-12-09_at_23.40.09.png|none|400px|weiboscope]]
*11/2018: Try running LSTM/tensorflow training with Python (following text predictor)from my PhD thesis text
[[File:Text predictor.png|none|400px|PhD thesis]]
*11/2018: Try running local ml5 + python training with English text (multiple manifestos) and generate 10000 characters text from multiple manifestos
[[File:Training.png|none|400px|manifestos]]
*06/2018: Try running LSTM ml5js with training simplied chinese data. Source from weiboscope 2012 week 1 deleted text
[[File:erasedText.jpg|none|400px|weiboscope text]]
*06/2018: Running LSTM ml5js example with my own training data
[[File:text_training.png|none|400px|Training Process]]
[[File:trained_text.png|none|400px|Outcome Process]]
*2018: Running ml5.js example - Simple LSTM Generator Example on a local browser
[[File:LSTMText.png|none|400px|Predicting what's the text]]
*Running ml5.js example - Simple Image Classification Example on a local browser
[[File:Ml5.png|none|400px|Predicting what's the image with confidence level]]
*Running spam data with RecurrentJS on a local browser
[[File:Learningspam.png|none|400px|Running spam data with RecurrentJS]]
*Running a customized neural network on a local browser
[[File:Neuralnetwork.png|none|400px|Learning XOR with Synaptic]]
*Running a PNG file with Synaptic.js on a local browser
[[File:synaptic.png|none|400px|Learning a png file with Synaptic]]
*Running a jpg file with ConvNetJS on a local browser
[[File:ML_JS_64.png|none|400px|Learning a jpg file with ConvNetJS]]

Latest revision as of 17:35, 12 January 2020

Introduction to Machine Learning

Cultural matters with Machine Learning

Technical explanation on Machine Learning

RNN/LSTM focus

Neural Network

Data Processing

Examples with source code

Mixed media (text/image)

Text related

Chinese text related

Image related

Sound related

Projects

Text related

Artworks

Examples/Performance/Speculative design

Exhibition

Workshop

Conference

Preconference Workshop] (2018)

Demo/Experimental Projects

Learning resource

Teaching Machine Learning

Experiments/Tests

  • 01/2020: TensorFlow.js + React, object detection
object
  • 08/2019: Runway's image to text prediction: "a collage of photos of a man on a motorcycle"
runway3
  • 08/2019: Runway's YOLACT - computer and algorithm shift our attention to something else. A lid cannot be recognized but that's the key object in the photo (ways of human seeing) actually. May be the training data contains mostly western objects.
    runway1
    runway2
  • 08/2019: Running LSTM/tensorflow with text_predictor.py and ml5 training
training both
  • 12/2018: Try running LSTM/tensorflow training on chinese text with python (again following text predictor)
weiboscope
  • 11/2018: Try running LSTM/tensorflow training with Python (following text predictor)from my PhD thesis text
PhD thesis
  • 11/2018: Try running local ml5 + python training with English text (multiple manifestos) and generate 10000 characters text from multiple manifestos
manifestos
  • 06/2018: Try running LSTM ml5js with training simplied chinese data. Source from weiboscope 2012 week 1 deleted text
weiboscope text
  • 06/2018: Running LSTM ml5js example with my own training data
Training Process
Outcome Process
  • 2018: Running ml5.js example - Simple LSTM Generator Example on a local browser
Predicting what's the text
  • Running ml5.js example - Simple Image Classification Example on a local browser
Predicting what's the image with confidence level
  • Running spam data with RecurrentJS on a local browser
Running spam data with RecurrentJS
  • Running a customized neural network on a local browser
Learning XOR with Synaptic
  • Running a PNG file with Synaptic.js on a local browser
Learning a png file with Synaptic
  • Running a jpg file with ConvNetJS on a local browser
Learning a jpg file with ConvNetJS