Category Archives: PYTHON

SUBSCRIBE TO UI STREET

PYTHON’S FLASK

Python’s Flask

Python_Flask
Flask is a small and powerful web framework for Python. It is easy to learn and simple to use, enabling the users to build their web app in less amount of time. Flask is also easy to get started with as a beginner because there is little boilerplate code for getting a simple app up and running. Flask backs extensions that can add application features as if they were implemented in Flask itself. Extensions exist for object-relational mappers, form validation, upload handling, and several common frameworks related tools. Extensions are updated more regularly than the core Flask program. Flask is commonly used with MongoDB which allows it more control over databases and history.

INSTALLING FLASK

Before getting started, the user need to install Flask. Because systems vary, things can intermittently go wrong during these steps.

INSTALL VIRTUALENV

Here we will be using virtualenv to install Flask. Virtualenv is a suitable tool that creates isolated Python development environments where the user can do all his/her development work. If the user installs it system-wide, there is the risk of messing up other libraries that the user might have installed already. Instead, use virtualenv to create a sandbox, where the user can install and use the library without affecting the rest of the system. The user can keep using sandbox for ongoing development work, or can simply delete it once the user is finished using it. Either way, the system remains organized and clutter-free.

If you see a version number, you are good to go and you can skip to this “Install Flask” section. If the command was not found, use easy_install or pip to install virtualenv. If you are running in Linux or Mac OS X, one of the following should work:

$ sudo easy_install virtualenv

$ sudo pip install virtualenv
If you are running Windows, follow the “Installation Instructions” on this page to get easy_install up and running on your system.

INSTALL FLASK

After installing virtualenv, the user can create a new isolated development environment, like so:

$ virtualenv flaskapp

Here, virtualenv creates a folder, flaskapp/, and sets up a clean copy of Python inside for the user to use. It also installs the handy package manager, pip.

Enter newly created development environment and activate it so to start working within it.

1
2
$ cd flaskapp
$ . bin/activate
Now, the user can safely install Flask:
$ pip install Flask

SETTING UP THE PROJECT STRUCTURE

Let’s create a couple of folders and files within flaskapp/ to keep the web app organized.

INSTALL FLASK

Within flaskapp/, create a folder, app/, to comprise all files. Inside app/, create a folder static/; this is where the user has to put the web app’s images, CSS, and JavaScript files, so create folders for each of those, as demonstrated above. As well, create another folder, templates/, to store the app’s web templates. Create an empty Python file routes.py for the application logic, such as URL routing.

And no project is complete without a helpful description, so create a README.md file as well.

BUILDING A HOME PAGE

While writing a web app with a couple of pages, it quickly becomes bothersome to write the same HTML boilerplate over and over again for each page. Also, if the user needs to add a new element to their application, such as a new CSS file, the user would have to go into every single page and should add. This is time consuming and error prone. Wouldn’t be nice if, instead of repeatedly writing the same HTML boilerplate, the user can define their page layout just once, and then use that layout to make new pages with their own content.

APP/TEMPLATES/HOME.HTML

1 {% extends “layout.html” %}

2 {% block content %}

3 <div class=”jumbo”>

4 <h2>Welcome to the Flask app<h2>

5 <h3>This is the home page for the Flask app<h3>

6 </div>

7 {% endblock %}

BUILDING AN ABOUT PAGE

In the above section, we have seen the creation of a web template home.html. Now, let’s repeat that process again to create an about page for our web app.

APP/TEMPLATES/ABOUT.HTML

{% extends “layout.html” %}

{% block content %}

<h2>About</h2>

<p>This is an About page for the Intro to Flask article. Don’t I look good? Oh stop, you’re making me blush.</p>

{% endblock %}

In order to visit this page in the browser, we need to map a URL to it. Open up routes.py and add another mapping:

from flask import Flask, render_template

app = Flask(__name__)

@app.route(‘/’)

def home():

return render_template(‘home.html’)

@app.route(‘/about’)

def about():

return render_template(‘about.html’)

if __name__ == ‘__main__':

app.run(debug=True)

SUBSCRIBE TO UI STREET

WHAT HAPPENS TO THE TOTAL MEMORY WHEN PYTHON EXISTS?

What happens to the total memory when python exists?

Whenever Python exits, especially those python modules which are having circular references to other objects or the objects that are referenced from the global namespaces are not always de – allocated/freed/uncollectable.

It is impossible to deallocate those portions of memory that are reserved by the C library.

On exit, because of having its own efficient clean up mechanism, Python would try to deallocate/destroy every object.

SUBSCRIBE TO UI STREET

SUBSCRIBE TO UI STREET

FLASK VS DJANGO

Flask Vs Django

Flask Vs Django

Flask and Django are two of the most popular web frameworks for Python. In this article, we will be discussing some of the points you should consider while choosing between Flask and Django.

WHAT IS DJANGO AND WHAT DOES IT DO?

Django is a well-sophisticated framework aimed at rapid deployment and development of numerous web apps written in Python. This framework is distributed as an open-source. The framework itself is actually a code library, which helps the developers in building reliable, scalable, and workable web apps. Django is one of the most popular frameworks from a wide variety, available to Python developers. There is one limitation though: some things are envisioned to be done in one and the only way.

You can replace certain modules yet some core functionality should remain untouched. This is totally fine in 95% of the projects, and it saves a ton of time, money, and effort during development, as the users will have all the solutions they need straight out of the box.

WHAT IS FLASK?

Flask is another widely used web framework. Differing to Django, it is focused on at providing an out-of-the-box product with complete solutions for each task, Flask works more like a LEGO set, where the user can construct anything he/she wish, using an enormous set of external libraries and add-ons. Flask philosophy is “web development, one drop at a time”. Python developers with huge experience say that Flask enables adding new modules when the time comes, instead of overwhelming the users with the details from the very beginning.

LET’S TAKE A CLOSER LOOK AT WHAT POSSIBILITIES DJANGO HAS TO OFFER

  • Object Relational Mapping allows working with several types of databases such as SQLite, Post greSQL, Oracle and MySQL.
  • Celery allows doing asynchronous tasks and replacing unix crontab for cron jobs.
    The user can use Gunicorn instead of Apache; it’s easy and fun (if the user has no trouble with using NGINX).
  • If the developers are more skilled enough, then the user can use MongoDB as a primary database; this solves quite a lot of problems later on.
  • Using named URLs, reverse function, and the URL template tag allows creating a logically structured system, where one URL update will not inflict confusion.
  • Using supervisor for process monitoring lets restart the framework processes automatically; it is truly a rescuer during development stage.
  • Redis is a valued in-memory data structure store, which can be used for queuing celery jobs, as a cache, as a store for sessions, even for auto-completion and much more.
  • Munin and statds are another great pair of apps, allowing control and monitoring of the users Django app processes.
  • As we can see from the list of websites using this framework, it is planned for creating apps with high scalability; websites that grow from thousands to millions of visitors quickly. This framework works straight out of the box and provides all the major functionalities needed to build an app with Python.

LET US TAKE A LOOK AT FLASK ARCHITECTURE AND FUNCTIONALITY

  • Flask is all about simplicity and ease. There are no limitations and the user can implement anything they want it.
  • No database access layer and ORM. Other apps like SQLAlchemy or pure SQL queries do this job without any restrictions.
  • Routing with decorators is really simple; app structure is also totally adjustable.
    Blueprints are like modules for the application. The user can have lots of them suited for any task and construct to their app like a LEGO toy, using the blueprints best suited for this particular task and the extensions are incredibly helpful and are integrated into the framework easily.
  • Web server and debugging tools: Flask comes with in-built web server and multiple debugging tools, including the in-browser, so the user do not even need NGINX or Apache to test and debug the app.
  • Flask appeared as a substitute to Django, as developers chosen to have a micro framework that would allow them using varying components, and neither of previous frameworks allowed alteration of their modules to some extent. Flask is simple and straightforward thus working in it allows an experienced Python developer in creating projects within short timeframes.

THE VERDICT

Flask works like a sandbox for developers, where they can improve their skills and quickly test solutions using different modules and libraries. We prefer using it for testing and working on less-structured objects, while using Django to deliver a solid product, meeting and exceeding customer’s expectations.

SUBSCRIBE TO UI STREET

HOW TO MERGE TWO DICTIONARIES IN A SINGLE EXPRESSION?

How to merge two dictionaries in a single expression?

For dictionaries x and y, z becomes a merged dictionary with values from y replacing those from x.
In Python 3.5 or greater, :
z = {**x, **y}
In Python 2, (or 3.4 or lower) write a function:
def merge_two_dicts(x, y):
z = x.copy() # start with x’s keys and values
z.update(y) # modifies z with y’s keys and values & returns None
return z
and
z = merge_two_dicts(x, y)

SUBSCRIBE TO UI STREET

MICROSOFT CONSIDERS ADDING PYTHON AS AN OFFICIAL SCRIPTING LANGUAGE TO EXCEL

Microsoft Considers Adding Python as an Official Scripting Language to Excel

According to a topic on Excel’s feedback hub published few days back, Microsoft considers adding Python to Excel. This topic has turned out to be the vastly voted feature request, ever since the news was on the hub.

PYTHON TURNS DIFFICULTY TO SIMPLICITY

Python is one of the most popular programming languages among developers, due to its easiness in coding and its flexibility. To know where it ranks while compared to other programming languages, Python ranks second on the PYPL programming languages ranking. According to the RedMonk Programming Language Rankings, it ranks third, and fourth in the TIOBE index.

If Python for Excel is approved by Microsoft, one can easily work with Excel documents, Excel data, and with its core functions, using Python scripts replacing the current VBA scripts.

This Python scripting, would not only turn out to be a replacement for VBA but also could be a replacement for field functions (=SUM(A1:A2)).

The idea of having Python as an official Excel scripting language was highly appreciated by many users on board. Moreover, the users also marked that if Microsoft goes forward in wiring Python within Excel, they also would require Python in other Microsoft Office apps.

After watching the news on the hub, many users and developers said that Microsoft should definitely choose Python and during this process, it should also decide whether it would be Python with a .Net library–which has separate standard and core libraries–or IronPython. Later, this has to be done in a process that provides exactly same libraries and user-written code to work in the same way throughout other Microsoft Office products.

Though Microsoft would be elated in adding such a feature for their users, still, not much information is known about this project. Until then we can expect great bombshells from Microsoft with user side benefits.

SUBSCRIBE TO UI STREET

REGULAR EXPRESSIONS IN PYTHON

Regular Expressions in Python

A regular expression is a distinctive sequence of characters that helps the user to match or find other strings or sets of strings, using a specialized syntax held in a pattern. Regular expressions are widely used in UNIX world. The module re offers full support for Perl-like regular expressions in Python.

In this article we will be covering important functions, which would be used to handle regular expressions.

IN PYTHON A REGULAR EXPRESSION SEARCH IS TYPICALLY WRITTEN AS

match = re.search(pat, str)

The re.search() method takes a regular expression pattern and a string and searches for that pattern within the string. If the search is successful, search() returns a match object or None. Therefore, the search is usually followed by an if-statement to test if the search is succeeded or not.

str = ‘an example word:cat!!’
match = re.search(r’word:\w\w\w’, str)
# If-statement after search() tests if it succeeded
if match:
print ‘found’, match.group() ## ‘found word:cat’
else:
print ‘did not find’

The code match = re.search(pat, str) stores the search result in a variable named “match”. Then the if-statement tests the match — if true the search succeeded and match.group() is the matching text (e.g. ‘word:cat’). Otherwise if the match is false, then the search will not be succeeded, and there is no matching text.

The ‘r’ at the start of the pattern string designates a python “raw” string which passes through backslashes without change which is very convenient for regular expressions.

if-statement tests the match — if true the search succeeded and match.group() is the matching text (e.g. ‘word:cat’). If the match is false, then the search will not be succeeded, and there is no matching text.

The ‘r’ at the start of the pattern string designates a python “raw” string which passes through backslashes without change which is very convenient for regular expressions.

THE MATCH FUNCTION

The function attempts to match re pattern to string with optional flags.

Here is the syntax for this function:
re.match(pattern, string, flags=0)

The re.match function returns a match object on success. The usegroup(num) or groups() function is used to match objects to get matched expression.

The match function

#!/usr/bin/python
import re
line = “Cats are smarter than dogs”
matchObj = re.match( r'(.*) are (.*?) .*’, line, re.M|re.I)
if matchObj:
print “matchObj.group() : “, matchObj. group ()
print “matchObj.group(1) : “, matchObj. group (1)
print “matchObj.group(2) : “, matchObj. group (2)
else:
print “No match!!”

When the above code is executed, it gives the following result:

matchObj.group() : Cats are smarter than dogs
matchObj.group(1) : Cats
matchObj.group(2) : smarter

SUBSCRIBE TO UI STREET

DATA MINING WITH PYTHON

Data Mining With Python

Data mining is the extraction of implicit, formerly unknown, and potentially useful information from data. It is applied in a wide range of domains and its practices have become fundamental for several applications.

This article is about the tools used in real Data Mining for finding and describing structural patterns in data using Python. In recent years, Python has been used for the development of data-centric.

DATA IMPORTING AND VISUALIZATION

The very first step of a data analysis consists of obtaining the data and loading the data into the user’s work environment. User can easily download data using the following Python capability:

import urllib2

url = ‘http://aima.cs.devopspython.edu/data/iris.csv‘

u = urllib2.urlopen(url)

localFile = open(‘iris.csv”, ‘w’)

localFile.write(u.read())

localFile.close()

In the above snippet user has used the library urllib2 to access a file on the website and saved it to the disk using the methods of the File object provided by the standard library. The file contains the iris dataset, which is a multivariate dataset that consists of 50 samples from each of three species of Iris flowers. Each sample has four features that is the length and the width of sepal and petal, in centimetres.

The dataset is stored in the CSV format. It is appropriate to parse the CSV file and to store the informa tion that it contains using a more suitable data structure. The dataset has 5 rows, the first 4 rows contain the values of the features while the last row signifies the class of the samples. The CSV can be easily parsed using the function genfromtxt of the numpy library:

from numpy import genfromtxt, zeros

# read the first 4 columns

data = genfromtxt(‘iris.csv’,delimiter=’,’,usecols=(0,1,2,3))

# read the fifth column

target = genfromtxt(‘iris.csv’,delimiter=’,’,usecols=(4),dtype=str)

In the above example user has created a matrix with the features and a vector that contains the classes. The user can also confirm the size of the dataset looking at the shape of the data structures loaded:

print data.shape

(150, 4)

print target.shape

(150,)

print set(target) # build a collection of unique elements

set([‘setosa’, ‘versicolor’, ‘virginica’])

An important task when working with a new data is to understand what information the data contains and how it is structured. Visualization helps the user to explore the information graphically in such a way to gain understanding and insight into the data.

CLASSIFICATION

Classification is a data mining function that allocates samples in a dataset to target classes. The models that implement this function are called classifiers. There are two basic steps for using a classifiers: training and classification. The library sklearn contains the implementation of many models for classification.

t = zeros(len(target))

t[target == ‘setosa’] = 1

t[target == ‘versicolor’] = 2

t[target == ‘virginica’] = 3

The classification can be done with the predict method and it is easy to test it with one of the sample:

print classifier.predict(data[0])

[ 1.]

print t[0]

1

In this case the predicted class is equal to the correct one (setosa), but it is important to assess the classifier on a wider range of samples and to test it with data not used in the training process.

CLUSTERING

We do not have labels attached to the data that tell us the class of the samples. The user has to analyse the data in order to group them on the basis of a similar criteria where groups are sets of similar samples. This kind of analysis is called unsupervised data analysis. One of the most famous clustering tools is the k-means algorithm, which can be run as follows:

from sklearn.cluster import KMeans

kmeans = KMeans(k=3, init=’random’) # initialization

kmeans.fit(data) # actual execution

The snippet above runs the algorithm and groups the data in 3 clusters (as specified by the parameter k). Now the user can use the model to assign each sample to one of the clusters:

c = kmeans.predict(data)

And the user can evaluate the results of clustering, comparing it with the labels that they already have using the completeness and the homogeneity of the score:

from sklearn.metrics import completeness_score, homogeneity_score

print completeness_score(t,c)

0.7649861514489815

print homogeneity_score(t,c)

0.7514854021988338

The wholeness of the score approaches 1 when most of the data points that are members of a given class are elements of the same cluster while the homogeneity score approaches 1 when all the clusters contain almost only data points that are member of a single class.

The user can also visualize the result of the clustering and compare the assignments with the real labels visually:

figure()

subplot(211) # top figure with the real classes

plot(data[t==1,0],data[t==1,2],’bo’)

plot(data[t==2,0],data[t==2,2],’ro’)

plot(data[t==3,0],data[t==3,2],’go’)

subplot(212) # bottom figure with classes assigned automatically

plot(data[c==1,0],data[tt==1,2],’bo’,alpha=.7)

plot(data[c==2,0],data[tt==2,2],’go’,alpha=.7)

plot(data[c==0,0],data[tt==0,2],’mo’,alpha=.7)

show()

SUBSCRIBE TO UI STREET

WHAT IS GARBAGE COLLECTION AND DOES PYTHON HAVE IT?

What is garbage collection and does python have it?

Garbage collection is the systematic recovery of pooled computer storage that is being used by a program when that program no longer needs the storage which frees the storage for use by other programs.

Python also have an inbuilt garbage collector, which recycles all the unused memory and frees the memory and makes it available to the heap space.