Friday, 30 November 2018
Machine Learning For Dummies
Table of Contents INTRODUCTION............................................................................................... 1 About This Book ................................................................................... 1 Foolish Assumptions............................................................................ 2 Icons Used in This Book....................................................................... 2 CHAPTER 1: Understanding Machine Learning................................. 3 What Is Machine Learning? ................................................................. 4 Iterative learning from data........................................................... 5 What’s old is new again.................................................................. 5 Defining Big Data.................................................................................. 6 Big Data in Context with Machine Learning...................................... 7 The Need to Understand and Trust your Data................................. 8 The Importance of the Hybrid Cloud................................................. 9 Leveraging the Power of Machine Learning ..................................... 9 Descriptive analytics.....................................................................10 Predictive analytics .......................................................................10 The Roles of Statistics and Data Mining with Machine Learning...............................................................................11 Putting Machine Learning in Context ..............................................12 Approaches to Machine Learning ....................................................14 Supervised learning......................................................................15 Unsupervised learning .................................................................15 Reinforcement learning ...............................................................16 Neural networks and deep learning...........................................17 CHAPTER 2: Applying Machine Learning ..............................................19 Getting Started with a Strategy.........................................................19 Using machine learning to remove biases from strategy........20 More data makes planning more accurate ...............................22 Understanding Machine Learning Techniques...............................22 Tying Machine Learning Methods to Outcomes ............................23 Applying Machine Learning to Business Needs..............................23 Understanding why customers are leaving...............................24 Recognizing who has committed a crime ..................................25 Preventing accidents from happening.......................................26 iv Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. CHAPTER 3: Looking Inside Machine Learning................................27 The Impact of Machine Learning on Applications..........................28 The role of algorithms..................................................................28 Types of machine learning algorithms.......................................29 Training machine learning systems............................................33 Data Preparation................................................................................34 Identify relevant data ...................................................................34 Governing data..............................................................................36 The Machine Learning Cycle .............................................................37 CHAPTER 4: Getting Started with Machine Learning.................39 Understanding How Machine Learning Can Help..........................39 Focus on the Business Problem .......................................................40 Bringing data silos together ........................................................41 Avoiding trouble before it happens............................................42 Getting customer focused ...........................................................43 Machine Learning Requires Collaboration......................................43 Executing a Pilot Project....................................................................44 Step 1: Define an opportunity for growth..................................44 Step 2: Conducting a pilot project...............................................44 Step 3: Evaluation .........................................................................45 Step 4: Next actions......................................................................45 Determining the Best Learning Model ............................................46 Tools to determine algorithm selection.....................................46 Approaching tool selection..........................................................47 CHAPTER 5: Learning Machine Skills .......................................................49 Defining the Skills That You Need ....................................................49 Getting Educated................................................................................53 IBM-Recommended Resources ........................................................56 CHAPTER 6: Using Machine Learning to Provide Solutions to Business Problems ....................................57 Applying Machine Learning to Patient Health ................................57 Leveraging IoT to Create More Predictable Outcomes..................58 Proactively Responding to IT Issues.................................................59 Protecting Against Fraud...................................................................60 CHAPTER 7: Ten Predictions on the Future of Machine Learning...............................................................63
Saturday, 30 December 2017
Deep Learning for Natural Language Processing
Contents III Data Preparation 34 IV BagofWords 61 V Word Embeddings 114 VI Text Classification 144 VII Language Modeling 189 VIII Image Captioning 244 IX Machine Translation 331 X Appendix 372 XI Conclusions 395 Copyright
Common terms and phrases approach architecture array bag-of-words better BLEU score calculate called caption chapter characters classification clean close Complete example convert create dataset deep learning define descriptions develop discover document encode Encoder-Decoder Epoch evaluate example Example output Explore extract file.close filename filter function given import input input sequence integer encode Keras labels language model layer length Listing load load doc load_doc(filename look loss mapping max_length means methods movie review natural language processing negative Neural Machine Translation neural network open(filename output pairs performance pre-trained predict prepare prints probability problem provides punctuation Python reference remove representation Running the example sentence sentiment sequence skill specific split started statistical step summarize task text data tokens turn tutorial vector vocab vocab_size vocabulary word embedding Word2Vec
Thursday, 31 July 2014
Natural Language Processing with Python [html edition]
Natural Language Processing with Python
– Analyzing Text with the Natural Language Toolkit
Steven Bird, Ewan Klein, and Edward Loper
This version of the NLTK book is updated for Python 3 and NLTK 3. The first edition of the book, published by O'Reilly, is available at http://nltk.org/book_1ed/. (There are currently no plans for a second edition of the book.)
0. Preface
1. Language Processing and Python
2. Accessing Text Corpora and Lexical Resources
3. Processing Raw Text
4. Writing Structured Programs
5. Categorizing and Tagging Words (minor fixes still required)
6. Learning to Classify Text
7. Extracting Information from Text
8. Analyzing Sentence Structure
9. Building Feature Based Grammars
10. Analyzing the Meaning of Sentences (minor fixes still required)
11. Managing Linguistic Data (minor fixes still required)
12. Afterword: Facing the Language Challenge
Bibliography
Term Index
Friday, 31 December 2010
HANDBOOK OF NATURAL LANGUAGE PROCESSING SECOND EDITION
PART I Classical Approaches
1 Classical Approaches to Natural Language Processing Robert Dale ................. 3
2 Text Preprocessing David D. Palmer.......................................................... 9
3 Lexical Analysis Andrew Hippisley............................................................. 31
4 Syntactic Parsing Peter Ljunglöf and Mats Wirén ......................................... 59
5 Semantic Analysis Cliff Goddard and Andrea C. Schalley ............................... 93
6 Natural Language Generation David D. McDonald ...................................... 121
PART II Empirical and Statistical Approaches
7 Corpus Creation Richard Xiao .................................................................. 147
8 Treebank Annotation Eva Hajicová, Anne Abeillé, Jan Haji ˇ c, Ji ˇ rí Mírovský, ˇ
and Zdenka Urešová ˇ .................................................................................. 167
9 Fundamental Statistical Techniques Tong Zhang ......................................... 189
10 Part-of-Speech Tagging Tunga Güngör ...................................................... 205
11 Statistical Parsing Joakim Nivre................................................................. 237
12 Multiword Expressions Timothy Baldwin and Su Nam Kim........................... 267
vii
viii Contents
13 Normalized Web Distance and Word Similarity Paul M.B. Vitányi
and Rudi L. Cilibrasi .................................................................................. 293
14 Word Sense Disambiguation David Yarowsky ............................................. 315
15 An Overview of Modern Speech Recognition Xuedong Huang and Li Deng ...... 339
16 Alignment Dekai Wu .............................................................................. 367
17 Statistical Machine Translation Abraham Ittycheriah.................................... 409
PART III Applications
18 Chinese Machine Translation Pascale Fung ................................................ 425
19 Information Retrieval Jacques Savoy and Eric Gaussier ................................. 455
20 Question Answering Diego Mollá-Aliod and José-Luis Vicedo ........................ 485
21 Information Extraction Jerry R. Hobbs and Ellen Riloff ................................. 511
22 Report Generation Leo Wanner ................................................................ 533
23 Emerging Applications of Natural Language Generation in Information
Visualization, Education, and Health Care Barbara Di Eugenio
and Nancy L. Green ................................................................................... 557
24 Ontology Construction Philipp Cimiano, Johanna Völker,
and Paul Buitelaar .................................................................................... 577
25 BioNLP: Biomedical Text Mining K. Bretonnel Cohen .................................. 605
26 Sentiment Analysis and Subjectivity Bing Liu .............................................. 627
Index ............................................................................................................ 667
https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/Handbook%20Of%20Natural%20Language%20Processing,%20Second%20Edition%20Chapman%20&%20Hall%20Crc%20Machine%20Learning%20&%20Pattern%20Recognition%202010.pdf
https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/Handbook%20Of%20Natural%20Language%20Processing,%20Second%20Edition%20Chapman%20&%20Hall%20Crc%20Machine%20Learning%20&%20Pattern%20Recognition%202010.pdf
The Oxford Handbook of Computational Linguistics 1st ed
Introduction 1
Part I Formal Foundations 9
1 Formal Language Theory 11
SHULY WINTNER
2 Computational Complexity in Natural Language 43
IAN PRATT-HARTMANN
3 Statistical Language Modeling 74
CIPRIAN CHELBA
4 Theory of Parsing 105
MARK-JAN NEDERHOF AND GIORGIO SATTA
Part II Current Methods 131
5 Maximum Entropy Models 133
ROBERT MALOUF
6 Memory-Based Learning 154
WALTER DAELEMANS AND ANTAL VAN DEN BOSCH
7 Decision Trees 180
HELMUT SCHMID
8 Unsupervised Learning and Grammar Induction 197
ALEXANDER CLARK AND SHALOM LAPPIN
9 Artificial Neural Networks 221
JAMES B. HENDERSON
“9781405155816_1_000” — 2010/5/14 — 16:54 — page viii — #8
viii Contents
10 Linguistic Annotation 238
MARTHA PALMER AND NIANWEN XUE
11 Evaluation of NLP Systems 271
PHILIP RESNIK AND JIMMY LIN
Part III Domains of Application 297
12 Speech Recognition 299
STEVE RENALS AND THOMAS HAIN
13 Statistical Parsing 333
STEPHEN CLARK
14 Segmentation and Morphology 364
JOHN A. GOLDSMITH
15 Computational Semantics 394
CHRIS FOX
16 Computational Models of Dialogue 429
JONATHAN GINZBURG AND RAQUEL FERNÁNDEZ
17 Computational Psycholinguistics 482
MATTHEW W. CROCKER
Part IV Applications 515
18 Information Extraction 517
RALPH GRISHMAN
19 Machine Translation 531
ANDY WAY
20 Natural Language Generation 574
EHUD REITER
21 Discourse Processing 599
RUSLAN MITKOV
22 Question Answering 630
BONNIE WEBBER AND NICK WEBB
References 655
Author Index 742
Subject Index 763
http://santini.se/teaching/sais/ClarkEtAl2010_HandbookNLP.pdf
http://santini.se/teaching/sais/ClarkEtAl2010_HandbookNLP.pdf
Friday, 1 October 2010
Introduction to Machine Learning
Introduction to Machine Learning
Alex Smola and S.V.N. Vishwanathan Yahoo! Labs Santa Clara –and– Departments of Statistics and Computer Science Purdue University –and– College of Engineering and Computer Science Australian National University
Contents Preface page 1 1 Introduction 3 1.1 A Taste of Machine Learning 3 1.1.1 Applications 3 1.1.2 Data 7 1.1.3 Problems 9 1.2 Probability Theory 12 1.2.1 Random Variables 12 1.2.2 Distributions 13 1.2.3 Mean and Variance 15 1.2.4 Marginalization, Independence, Conditioning, and Bayes Rule 16 1.3 Basic Algorithms 20 1.3.1 Naive Bayes 22 1.3.2 Nearest Neighbor Estimators 24 1.3.3 A Simple Classifier 27 1.3.4 Perceptron 29 1.3.5 K-Means 32 2 Density Estimation 37 2.1 Limit Theorems 37 2.1.1 Fundamental Laws 38 2.1.2 The Characteristic Function 42 2.1.3 Tail Bounds 45 2.1.4 An Example 48 2.2 Parzen Windows 51 2.2.1 Discrete Density Estimation 51 2.2.2 Smoothing Kernel 52 2.2.3 Parameter Estimation 54 2.2.4 Silverman’s Rule 57 2.2.5 Watson-Nadaraya Estimator 59 2.3 Exponential Families 60 2.3.1 Basics 60 v vi 0 Contents 2.3.2 Examples 62 2.4 Estimation 66 2.4.1 Maximum Likelihood Estimation 66 2.4.2 Bias, Variance and Consistency 68 2.4.3 A Bayesian Approach 71 2.4.4 An Example 75 2.5 Sampling 77 2.5.1 Inverse Transformation 78 2.5.2 Rejection Sampler 82 3 Optimization 91 3.1 Preliminaries 91 3.1.1 Convex Sets 92 3.1.2 Convex Functions 92 3.1.3 Subgradients 96 3.1.4 Strongly Convex Functions 97 3.1.5 Convex Functions with Lipschitz Continous Gradient 98 3.1.6 Fenchel Duality 98 3.1.7 Bregman Divergence 100 3.2 Unconstrained Smooth Convex Minimization 102 3.2.1 Minimizing a One-Dimensional Convex Function 102 3.2.2 Coordinate Descent 104 3.2.3 Gradient Descent 104 3.2.4 Mirror Descent 108 3.2.5 Conjugate Gradient 111 3.2.6 Higher Order Methods 115 3.2.7 Bundle Methods 121 3.3 Constrained Optimization 125 3.3.1 Projection Based Methods 125 3.3.2 Lagrange Duality 127 3.3.3 Linear and Quadratic Programs 131 3.4 Stochastic Optimization 135 3.4.1 Stochastic Gradient Descent 136 3.5 Nonconvex Optimization 137 3.5.1 Concave-Convex Procedure 137 3.6 Some Practical Advice 139 4 Online Learning and Boosting 143 4.1 Halving Algorithm 143 4.2 Weighted Majority 144 Contents vii 5 Conditional Densities 149 5.1 Logistic Regression 150 5.2 Regression 151 5.2.1 Conditionally Normal Models 151 5.2.2 Posterior Distribution 151 5.2.3 Heteroscedastic Estimation 151 5.3 Multiclass Classification 151 5.3.1 Conditionally Multinomial Models 151 5.4 What is a CRF? 152 5.4.1 Linear Chain CRFs 152 5.4.2 Higher Order CRFs 152 5.4.3 Kernelized CRFs 152 5.5 Optimization Strategies 152 5.5.1 Getting Started 152 5.5.2 Optimization Algorithms 152 5.5.3 Handling Higher order CRFs 152 5.6 Hidden Markov Models 153 5.7 Further Reading 153 5.7.1 Optimization 153 6 Kernels and Function Spaces 155 6.1 The Basics 155 6.1.1 Examples 156 6.2 Kernels 161 6.2.1 Feature Maps 161 6.2.2 The Kernel Trick 161 6.2.3 Examples of Kernels 161 6.3 Algorithms 161 6.3.1 Kernel Perceptron 161 6.3.2 Trivial Classifier 161 6.3.3 Kernel Principal Component Analysis 161 6.4 Reproducing Kernel Hilbert Spaces 161 6.4.1 Hilbert Spaces 163 6.4.2 Theoretical Properties 163 6.4.3 Regularization 163 6.5 Banach Spaces 164 6.5.1 Properties 164 6.5.2 Norms and Convex Sets 164 7 Linear Models 165 7.1 Support Vector Classification 165 viii 0 Contents 7.1.1 A Regularized Risk Minimization Viewpoint 170 7.1.2 An Exponential Family Interpretation 170 7.1.3 Specialized Algorithms for Training SVMs 172 7.2 Extensions 177 7.2.1 The ν trick 177 7.2.2 Squared Hinge Loss 179 7.2.3 Ramp Loss 180 7.3 Support Vector Regression 181 7.3.1 Incorporating General Loss Functions 184 7.3.2 Incorporating the ν Trick 186 7.4 Novelty Detection 186 7.5 Margins and Probability 189 7.6 Beyond Binary Classification 189 7.6.1 Multiclass Classification 190 7.6.2 Multilabel Classification 191 7.6.3 Ordinal Regression and Ranking 192 7.7 Large Margin Classifiers with Structure 193 7.7.1 Margin 193 7.7.2 Penalized Margin 193 7.7.3 Nonconvex Losses 193 7.8 Applications 193 7.8.1 Sequence Annotation 193 7.8.2 Matching 193 7.8.3 Ranking 193 7.8.4 Shortest Path Planning 193 7.8.5 Image Annotation 193 7.8.6 Contingency Table Loss 193 7.9 Optimization 193 7.9.1 Column Generation 193 7.9.2 Bundle Methods 193 7.9.3 Overrelaxation in the Dual 193 7.10 CRFs vs Structured Large Margin Models 194 7.10.1 Loss Function 194 7.10.2 Dual Connections 194 7.10.3 Optimization 194 Appendix 1 Linear Algebra and Functional Analysis 197 Appendix 2 Conjugate Distributions 201 Appendix 3 Loss Functions 203 Bibliography 221
Alex Smola and S.V.N. Vishwanathan Yahoo! Labs Santa Clara –and– Departments of Statistics and Computer Science Purdue University –and– College of Engineering and Computer Science Australian National University
Contents Preface page 1 1 Introduction 3 1.1 A Taste of Machine Learning 3 1.1.1 Applications 3 1.1.2 Data 7 1.1.3 Problems 9 1.2 Probability Theory 12 1.2.1 Random Variables 12 1.2.2 Distributions 13 1.2.3 Mean and Variance 15 1.2.4 Marginalization, Independence, Conditioning, and Bayes Rule 16 1.3 Basic Algorithms 20 1.3.1 Naive Bayes 22 1.3.2 Nearest Neighbor Estimators 24 1.3.3 A Simple Classifier 27 1.3.4 Perceptron 29 1.3.5 K-Means 32 2 Density Estimation 37 2.1 Limit Theorems 37 2.1.1 Fundamental Laws 38 2.1.2 The Characteristic Function 42 2.1.3 Tail Bounds 45 2.1.4 An Example 48 2.2 Parzen Windows 51 2.2.1 Discrete Density Estimation 51 2.2.2 Smoothing Kernel 52 2.2.3 Parameter Estimation 54 2.2.4 Silverman’s Rule 57 2.2.5 Watson-Nadaraya Estimator 59 2.3 Exponential Families 60 2.3.1 Basics 60 v vi 0 Contents 2.3.2 Examples 62 2.4 Estimation 66 2.4.1 Maximum Likelihood Estimation 66 2.4.2 Bias, Variance and Consistency 68 2.4.3 A Bayesian Approach 71 2.4.4 An Example 75 2.5 Sampling 77 2.5.1 Inverse Transformation 78 2.5.2 Rejection Sampler 82 3 Optimization 91 3.1 Preliminaries 91 3.1.1 Convex Sets 92 3.1.2 Convex Functions 92 3.1.3 Subgradients 96 3.1.4 Strongly Convex Functions 97 3.1.5 Convex Functions with Lipschitz Continous Gradient 98 3.1.6 Fenchel Duality 98 3.1.7 Bregman Divergence 100 3.2 Unconstrained Smooth Convex Minimization 102 3.2.1 Minimizing a One-Dimensional Convex Function 102 3.2.2 Coordinate Descent 104 3.2.3 Gradient Descent 104 3.2.4 Mirror Descent 108 3.2.5 Conjugate Gradient 111 3.2.6 Higher Order Methods 115 3.2.7 Bundle Methods 121 3.3 Constrained Optimization 125 3.3.1 Projection Based Methods 125 3.3.2 Lagrange Duality 127 3.3.3 Linear and Quadratic Programs 131 3.4 Stochastic Optimization 135 3.4.1 Stochastic Gradient Descent 136 3.5 Nonconvex Optimization 137 3.5.1 Concave-Convex Procedure 137 3.6 Some Practical Advice 139 4 Online Learning and Boosting 143 4.1 Halving Algorithm 143 4.2 Weighted Majority 144 Contents vii 5 Conditional Densities 149 5.1 Logistic Regression 150 5.2 Regression 151 5.2.1 Conditionally Normal Models 151 5.2.2 Posterior Distribution 151 5.2.3 Heteroscedastic Estimation 151 5.3 Multiclass Classification 151 5.3.1 Conditionally Multinomial Models 151 5.4 What is a CRF? 152 5.4.1 Linear Chain CRFs 152 5.4.2 Higher Order CRFs 152 5.4.3 Kernelized CRFs 152 5.5 Optimization Strategies 152 5.5.1 Getting Started 152 5.5.2 Optimization Algorithms 152 5.5.3 Handling Higher order CRFs 152 5.6 Hidden Markov Models 153 5.7 Further Reading 153 5.7.1 Optimization 153 6 Kernels and Function Spaces 155 6.1 The Basics 155 6.1.1 Examples 156 6.2 Kernels 161 6.2.1 Feature Maps 161 6.2.2 The Kernel Trick 161 6.2.3 Examples of Kernels 161 6.3 Algorithms 161 6.3.1 Kernel Perceptron 161 6.3.2 Trivial Classifier 161 6.3.3 Kernel Principal Component Analysis 161 6.4 Reproducing Kernel Hilbert Spaces 161 6.4.1 Hilbert Spaces 163 6.4.2 Theoretical Properties 163 6.4.3 Regularization 163 6.5 Banach Spaces 164 6.5.1 Properties 164 6.5.2 Norms and Convex Sets 164 7 Linear Models 165 7.1 Support Vector Classification 165 viii 0 Contents 7.1.1 A Regularized Risk Minimization Viewpoint 170 7.1.2 An Exponential Family Interpretation 170 7.1.3 Specialized Algorithms for Training SVMs 172 7.2 Extensions 177 7.2.1 The ν trick 177 7.2.2 Squared Hinge Loss 179 7.2.3 Ramp Loss 180 7.3 Support Vector Regression 181 7.3.1 Incorporating General Loss Functions 184 7.3.2 Incorporating the ν Trick 186 7.4 Novelty Detection 186 7.5 Margins and Probability 189 7.6 Beyond Binary Classification 189 7.6.1 Multiclass Classification 190 7.6.2 Multilabel Classification 191 7.6.3 Ordinal Regression and Ranking 192 7.7 Large Margin Classifiers with Structure 193 7.7.1 Margin 193 7.7.2 Penalized Margin 193 7.7.3 Nonconvex Losses 193 7.8 Applications 193 7.8.1 Sequence Annotation 193 7.8.2 Matching 193 7.8.3 Ranking 193 7.8.4 Shortest Path Planning 193 7.8.5 Image Annotation 193 7.8.6 Contingency Table Loss 193 7.9 Optimization 193 7.9.1 Column Generation 193 7.9.2 Bundle Methods 193 7.9.3 Overrelaxation in the Dual 193 7.10 CRFs vs Structured Large Margin Models 194 7.10.1 Loss Function 194 7.10.2 Dual Connections 194 7.10.3 Optimization 194 Appendix 1 Linear Algebra and Functional Analysis 197 Appendix 2 Conjugate Distributions 201 Appendix 3 Loss Functions 203 Bibliography 221
Monday, 1 June 2009
Natural Language Processing with Python
Chapter 1 Language Processing and Python
1
Chapter 2 Accessing Text Corpora and Lexical Resources
39
Chapter 3 Processing Raw Text
79
Chapter 4 Writing Structured Programs
129
Chapter 5 Categorizing and Tagging Words
179
Chapter 6 Learning to Classify Text
221
Chapter 7 Extracting Information from Text
261
Chapter 8 Analyzing Sentence Structure
291
Chapter 9 Building FeatureBased Grammars
327
Chapter 10 Analyzing the Meaning of Sentences
361
Chapter 11 Managing Linguistic Data
407
The Language Challenge
441
Bibliography
449
NLTK Index
459
General Index
463
Copyright
https://www.datascienceassn.org/sites/default/files/Natural%20Language%20Processing%20with%20Python.pdf
https://www.datascienceassn.org/sites/default/files/Natural%20Language%20Processing%20with%20Python.pdf
Subscribe to:
Posts (Atom)