Intel이 최적화 Library를 공급해오던것은 오래됐는데 Python쪽에도 지원하고 있는지 몰랐네요. 기사를 읽다보니 MS가 R을 2015년에 인수했었네요. 그래서 open source화 하고. 이것도 대단한 수인데요.

R, Python, SAS, MatLab, Mathematica

-----------------

The still-in-beta Python distribution uses Math Kernel Library to speed up processing on Intel hardware

http://www.infoworld.com/article/3044512/application-development/intels-python-distribution-provides-a-major-math-boost.html

-----------------

Rather than stuff Revolution R Open into a proprietary box, Microsoft is keeping the popular math-and-stats language open source under the GPL

http://www.infoworld.com/article/3021924/application-development/hooray-microsofts-r-will-remain-free-and-open.html

https://mran.revolutionanalytics.com/download/#download

-----------------

http://www.infoworld.com/article/2874853/big-data/microsoft-to-acquire-r-language-powerhouse-revolution-analytics.html

가끔식 로또할때가 있어서 그냥 무식하게 random으로 찍기는 싫어서 약간의 custom number를 만들수 있도록 간단히 만들어봄.

백만개씩 만들어놓고 당첨번호랑 매칭하는 것도 만들 예정임. 일주일에 백만개씩 사도 로또에 담청될수 없다는 교훈을 가르칠기 위해. ^^

----------------------

import random
import time
#TODO: use current time as random seed

lotterynumbers = range(1,71) # 1-70

# previous winning numbers
# high numbers
# not luckey numbers
exclude_numbers = [12,27,29,43,68,
				66,67,69,70, #68s
				4,22,53]

#print lotterynumbers

for num in exclude_numbers:
	lotterynumbers.remove(num)

print "Number Pool: %s" %lotterynumbers
print "------------------------"

x = 0
while x < 10:
	#lotterynumbers.append(random.randint(1, 10))
	#lotterynumbers.append(random.sample(range(10), 6))
	#lotterynumbers.sort()
	fredarray = random.sample(lotterynumbers, 5)
	fredarray.sort()
	print fredarray
	#print lotterynumbers
	x += 1
print;print

#Mega Number - one lucky number
mega_numbers = range(1,26)  # 1-25
mega_exclude_numbers = [1, 2, 22, 25]

for num in mega_exclude_numbers:
	mega_numbers.remove(num)
print "Mega Number Pool: %s" %mega_numbers
print "------------------------"
y = 0
while y < 5:
	print random.sample(mega_numbers, 1)
	y += 1

#end of the program

----------------------

random_numbers.py

-------------------

*USGS Json data structure

http://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson_detail.php

*Past Day M2.5+ Earthquakes

http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_day.geojson

-------------------

# 

import urllib2
import json

def printResults(data):
  # Use the json module to load the string data into a dictionary
  theJSON = json.loads(data)
  
  # now we can access the contents of the JSON like any other Python object
  if "title" in theJSON["metadata"]:
    print theJSON["metadata"]["title"]
  
  # output the number of events, plus the magnitude and each event name  
  count = theJSON["metadata"]["count"];
  print "Yesterday Total " + str(count) + " events recorded"
  
  # for each event, print the place where it occurred
#  for i in theJSON["features"]:
#    print i["properties"]["place"]

  # print the events that only have a magnitude greater than 4
  print"\n"
  print "Magnitude greater than 4:\n"
  for i in theJSON["features"]:
    if i["properties"]["mag"] >= 4.0:
      print "%2.1f" % i["properties"]["mag"], i["properties"]["place"]

  # print only the events where at least 1 person reported feeling something
  print "\n"
  print "Events that were felt:\n"
  for i in theJSON["features"]:
    feltReports = i["properties"]["felt"]
    placeOccur = i["properties"]["place"]
    if (feltReports != None) & (feltReports > 0):
        if (i["properties"]["place"].find("California") > 0 or i["properties"]["place"].find("Mexico") > 0) :
		    print "*****************************************"
        print "%2.1f" % i["properties"]["mag"], i["properties"]["place"], " reported " + str(feltReports) + " times"
 
  
def main():
  # define a variable to hold the source URL
  # In this case we'll use the free data feed from the USGS
  # This feed lists all earthquakes for the last day larger than Mag 2.5
  #Fred: http://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php
  urlData = "http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_day.geojson"
  
  # Open the URL and read the data
  webUrl = urllib2.urlopen(urlData)
  print ("USGS is alive, response=%d" % webUrl.getcode())
  if (webUrl.getcode() == 200):
    data = webUrl.read()
    # print out our customized results
    printResults(data)
  else:
    print "Received an error from server, cannot retrieve results " + str(webUrl.getcode())

if __name__ == "__main__":
  main()

-------------------

sample file

jsondata_finished.py

주식시세 query

webservicex는 response가 간단해서 빠른편인데, data 자체는 realtime도 아니고 15-20분에 한번씩 update는것 같음.

Google finance는 response자체가 너무 복잡함. 시간이 좀 걸림. 개인적인 용도로는 상관없지만.

간단하게 빠르게 update되는 서버는 무료는 없을듯. Nasdaq에서 한달에 얼마씩 내면 준다고 함.

-------------

http://www.webservicex.net/stockquote.asmx/GetQuote?symbol=aapl

XML response example)

<string xmlns="http://www.webserviceX.NET/">

<StockQuotes>

<Stock>

<Symbol>aapl</Symbol>

<Last>109.06</Last>

<Date>9/29/2015</Date>

<Time>4:00pm</Time>

<Change>-3.38</Change>

<Open>112.79</Open>

<High>113.51</High>

<Low>107.86</Low>

<Volume>73365384</Volume>

<MktCap>621.94B</MktCap>

<PreviousClose>112.44</PreviousClose>

<PercentageChange>-3.01%</PercentageChange>

<AnnRange>92.00 - 134.54</AnnRange>

<Earns>8.65</Earns>

<P-E>12.62</P-E>

<Name>Apple Inc.</Name>

</Stock>

</StockQuotes>

*but from urllib2 read, all tags are UTF-9 encoded(llike corrupted) like below ==> HTML parser로 escape문자를 바꿔줘야함

<?xml version="1.0" encoding="utf-8"?>

<string xmlns="http://www.webserviceX.NET/">&lt;StockQuotes&gt;&lt;Stock&gt;&lt;Symbol&gt;AAPL&lt;/Symbol&gt;&lt;Last&gt

;109.06&lt;/Last&gt;&lt;Date&gt;9/29/2015&lt;/Date&gt;&lt;Time&gt;4:00pm&lt;/Time&gt;&lt;Change&gt;-3.38&lt;/Change&gt;&

lt;Open&gt;112.79&lt;/Open&gt;&lt;High&gt;113.51&lt;/High&gt;&lt;Low&gt;107.86&lt;/Low&gt;&lt;Volume&gt;73365384&lt;/Vol

ume&gt;&lt;MktCap&gt;621.94B&lt;/MktCap&gt;&lt;PreviousClose&gt;112.44&lt;/PreviousClose&gt;&lt;PercentageChange&gt;-3.0

1%&lt;/PercentageChange&gt;&lt;AnnRange&gt;92.00 - 134.54&lt;/AnnRange&gt;&lt;Earns&gt;8.65&lt;/Earns&gt;&lt;P-E&gt;12.6

2&lt;/P-E&gt;&lt;Name&gt;Apple Inc.&lt;/Name&gt;&lt;/Stock&gt;&lt;/StockQuotes&gt;</string>

-------------

Pattern matching in XML

import HTMLParser
def fetchStockPrice(ticker):
	url="http://www.webservicex.net/stockquote.asmx/GetQuote?symbol="
	txt=urllib2.urlopen(url+ticker).read()
	html_parser = HTMLParser.HTMLParser()
	parsed = html_parser.unescape(txt)
	#print parsed
	#unicode =  txt.encode('utf-8')
	#print unicode

	#result=re.search('(.*?)',parsed)
	result=re.search('([0-9]+\.[0-9]+)', parsed)
	if result:
		stockprice = result.group(1)
		cur_date = re.search('(.*?)',parsed)
		cur_time = re.search('(.*?)',parsed)
		ret = stockprice + " " + cur_date.group(1) + " " + cur_time.group(1)
	else:
		stockprice = "Nothing found for: " + ticker
		ret = "Nothing found for: " + ticker
	return ret

print ("NOW = " + time.ctime())
print "Stock Price is " + fetchStockPrice("AAPL") 

-------------

import  urllib2, time, os, re
def fetchStockPrice(ticker):
	url="http://www.webservicex.net/stockquote.asmx/GetQuote?symbol="
	txt=urllib2.urlopen(url+ticker).read()
	#f=open("sample.xml")
	#txt = f.read()
	#print txt
	#result=re.search('(.*?)',txt)
	result=re.search('([0-9]+\.[0-9]+)',txt)
	#result=re.search("Last\>(\d+\.\d+)\</Last", txt)
	if result:
		stockprice = result.group(1)
	else:
		stockprice = "Nothing found for: " + ticker
	return stockprice

print(time.ctime())
print fetchStockPrice("AAPL") 

-------------

XML parsing example

https://docs.python.org/2/library/xml.dom.minidom.html?highlight=xml.dom.minidom#module-xml.dom.minidom

import xml.dom.minidom

document = """\

	
		aapl
		109.06
		9/29/2015
		4:00pm
		-3.38
		112.79
		113.51
		107.86
		73365384
		621.94B
		112.44
		-3.01%
		92.00 - 134.54
		8.65
		12.62
		Apple Inc.
	

"""

def getText(nodelist):
    rc = []
    for node in nodelist:
        if node.nodeType == node.TEXT_NODE:
            rc.append(node.data)
    return ''.join(rc)

def getStockPrice(quote):
	 print "Last Stock Price is :" % getText(quote.childNodes)

def handleToc(stocks):
    for stock in stocks:
		symbol = stock.getElementsByTagName("Symbol")[0]
		print "
%s
" % getText(symbol.childNodes)

		last = stock.getElementsByTagName("Last")[0]
		print "
%s
" % getText(last.childNodes)

		date = stock.getElementsByTagName("Date")[0]
		print "
%s
" % getText(date.childNodes)

		time = stock.getElementsByTagName("Time")[0]
		print "
%s
" % getText(time.childNodes)

dom = xml.dom.minidom.parseString(document)
stocks = dom.getElementsByTagName("Stock")
handleToc(stocks)

-------------

Stock Quote from Google finance

import urllib2, time, os, re, csv
 
def fetchGF(googleticker):
	url="http://www.google.com/finance?&q="
	txt=urllib2.urlopen(url+googleticker).read()
	#print txt
	#search phase 112.44
	k=re.search('id="ref_(.*?)">(.*?)<',txt)
	#print k
	if k:
		tmp1=k.group(0)
		tmp2=k.group(1)
		#print tmp1
		#print tmp2
		tmp=k.group(2)
		#print tmp
		q=tmp.replace(',','')
		#print q
	else:
		q="Nothing found for: "+googleticker
	return q
	
print(time.ctime())
print fetchGF("AAPL")
#print fetchGF("NASDAQ:AAPL")

-------------

Stock Portfolio를 query하려면

#file에서 읽어도 좋겠다.
tickers=["NASDAQ:AAPL","NASDAQ:GOOG","NASDAQ:BIDU","NYSE:IBM", \
         "NASDAQ:INTC","NASDAQ:MSFT","NYSEARCA:SPY"]

for ticker in tickers:
	data=combine(ticker)
	print(data)

-------------

-------------

import urllib2, time, os, re
import HTMLParser
def fetchStockPrice(ticker):
    url="http://www.webservicex.net/stockquote.asmx/GetQuote?symbol="
    txt=urllib2.urlopen(url+ticker).read()
    html_parser = HTMLParser.HTMLParser()
    parsed = html_parser.unescape(txt)
    #print parsed
    #unicode =  txt.encode('utf-8')
    #print unicode
 
    #result=re.search('(.*?)',parsed)
    result=re.search('([0-9]+\.[0-9]+)', parsed)
    if result:
        stockprice = result.group(1)
        cur_date = re.search('(.*?)',parsed)
        cur_time = re.search('(.*?)',parsed)
        cur_percent = re.search('(.*?)',parsed)
        #Text formatting
        ret = '{:4s}: {:7s} {:7s} - {:10s} {:7s}'.format(ticker, stockprice, cur_percent.group(1), cur_date.group(1), cur_time.group(1))
        #ret =  '{:4s}'.format(ticker) + ": " +stockprice + " " + cur_percent.group(1) + " " + cur_date.group(1) + " " + cur_time.group(1)
    else:
        stockprice = "Nothing found for: " + ticker
        ret = "Nothing found for: " + ticker
    return ret
 
print(time.ctime())

#tickers=["NASDAQ:AAPL","NASDAQ:GOOG","NASDAQ:QCOM","NYSE:NUAN", \
#         "NASDAQ:INTC","NYSE:DAL","NYSE:F", "NYSE:SMI", "NASDAQ:HQCL", "NYSEARCA:UCO"]
tickers=["AAPL","GOOG","QCOM","NUAN", \
         "INTC","DAL","F", "SMI", "HQCL", "UCO"]
print ("--------------------------")
#print fetchStockPrice("AAPL")
for ticker in tickers:
    data=fetchStockPrice(ticker)
    print(data)
print ("--------------------------") 

-------------

stockprices.py

Python으로 website를 돌리는것을 배우려고 강좌를 하나 봤는데 꽤 재밌고 빠르고 짧다.

문제는 python language가 그렇듯 high level abstraction이 쉬운것처럼 보이지만 반대로 상당히 이해하기 어렵다. 말은 쉬운데 진짜로 만들기 어려운것 처럼. 하여간 output을 만들면 짧고 간견하세 maintenance는 쉬울듯.

이번에 Python으로 inventory check하는 프로그램을 만들려고 하는데 이것으로 하면 재밌겠다.

------------

Up and Running with Python and Django

http://www.lynda.com/Developer-Web-Development-tutorials/Up-Running-Python-Django/386287-2.html

https://www.djangoproject.com/

http://djangogirls.com

*Need pip

sudo apt-get install python-pip

django-admin startproject <project_name>

manage.py runserver

*FIles

__init__.py

wggi.py : hook for web

urls.py

settings.py : DEBUG, Database

- Models - data layer

model.py

-Migration

manage.py

manage.py migrate --list

manage.py makemigration

manage.py migrate

*SQL browser

http://sqlitebrowser.org/ 

 URL Patterns  -> Views -> Templates

                                  |

                                  |

                                  |

                              Models

*New text editor

http://www.sublimetext.com/2

------------

Barcode scanner

https://play.google.com/store/apps/details?id=com.google.zxing.client.android&hl=en

https://github.com/zxing/zxing

ZXing-based third-party open source projects

Other related third-party open source projects

2009년 판인데 좀더 update하면 많은 사람들이 재밌어 할것 같다는 생각이 든다.

http://www.nltk.org/book/ch01.html

가면 온라인에서 책도 볼수 있습니다.

----------------------

INDEX

Pydub 

Pydub lets you do stuff to audio in a way that isn't stupid.

pip install pydub

git clone https://github.com/jiaaro/pydub.git

# libav
apt-get install libav-tools libavcodec-extra-53

####    OR    #####

# ffmpeg
apt-get install ffmpeg libavcodec-extra-53

pip install audiogrep

# transcribes all mp3s in the selected folder
audiogrep --input path/to/*.mp3 --transcribe

# returns all phrases with the word 'word' in them
audiogrep --input path/to/*.mp3 --search 'word'

sudo apt-get install python-pip

1. pip is a package management system used to install and manage software packages written in Python. Many packages can be found in the Python Package Index (PyPI).

Youtube에서 Python을 찾으니 아주 인기있는 speaker가 있는데, 얼굴도 예쁘고 거기다 금발까지. 거기다가 공부도 잘해. 완전 엄친아. Startup회사도 만들고 벌써 다른 회사에 팔아 넘기기까지(=돈도 많아) ^^ 파이썬의 얼굴이라고 해도 되겠네요.

Jessica McKellar

I am a startup founder, software engineer, and open source developer living in San Francisco, California.

http://web.mit.edu/jesstess/

-----------------------

한 video를 보게됐는데 프로그래밍 언어의 점유율에 대해서 잘 보여주고 있습니다. Github프로젝트로 가장 많은 수를 차지하는 언어는 JavaScript입니다. 이전에 포스팅했던 Popcorn Time과 NPM이런것들이 JavaScript언어로 되어 있음에 깜짝놀랐다는 글이었죠. (NPM engine은 C++입니다만) 

JAVA는 꾸준히 인기있는 언어이고요.

Perl은 죽었다고 봐야하겠고요.

PHP가 web programming에서는 꾸준히 있기를 유지하고 있네요.

JavaScript(JS)도 꾸준히 유지되네요.

Python은 폭발적으로 증가하고 있고요.

-----------------------

그러면서 티스토리 IT블로그에서 본 어떤 개발자들의 말이 생각났습니다.

누가 C/C++를 요즘 공부하냐는 말, C++는 이미 죽었다는 말. 점유율이나 대다수 사람들이 공부하는 추세를 보면 C/C++를 하는 사람이 없지요. 너무 어렵고 앞으로 사용하는 분야가 너무 없다고 생각되지요.

맞는 말이지만, 어떤 engine을 생각할때 저 휘에 있는 언어로 만들어서 팔수있는게 있습니까? 대다수의 회사는 C/C++로 engine/OS/Framework을 만들어 팝니다(or open source로 공개합니다.) 저 위에 언어는 새로운 새대의 언어이지만 UI에 가까워있는 일만 할수 있습니다. 좀더 장기적으로 오래가는 일을 하려면 그 아래의 일도 하는게 좋겠지요. 그래서 젊은 개발자들은 C/C++를 계속 배우는게 자신의 미래를 대비하는데 좋을거라 생각이 듭니다.(물론 하나도 잘 하기 힘든데라고 생각하면 하나에 집중해야겠지요)

젊은 개발자들이 저 위에 있는 언어만 한다면 이미 시장에서 잔뼈가 굵은 사람들의 job은 오래동안 상당히 안정적일것이라 생각이 됩니다. 우리는 좋은 세상에 태어난 것을 고맙게 생각하며 은퇴할수 있을까 궁금하네요.

-----------------------