Photon

Photon can extract the following data while crawling:

URLs (in-scope & out-of-scope)
URLs with parameters (example.com/gallery.php?id=2)
Intel (emails, social media accounts, amazon buckets etc.)
Files (pdf, png, xml etc.)
Secret keys (auth/API keys & hashes)
JavaScript files & Endpoints present in them
Strings matching custom regex pattern
Subdomains & DNS related data
The extracted information is saved in an organized manner or can be exported as json.

Python Regular Expression

A Regular Expression, is a sequence of characters that forms a search pattern. Python has a built-in package called re, which can be used to work with Regular Expressions.

(Example Code to remove symbols and numbers - compatible with Python 2.7.17)
import re
import string
input_str = "58597884|01:31:50|The rise of python stated by pythonforengineers blog"
print"Before Processing:",input_str
result = re.sub(r'\d+', '', input_str)
result = result.translate(string.maketrans("",""), string.punctuation)
print"After Processing:",result

More Details

Python Open Source Project - Manim

Manim is an animation engine for explanatory math videos. It's used to create precise animations programmatically. 

Install it from PyPI via pip:        pip3 install manimlib

Click here for more info

Surprise - A Python scikit for recommender systems

Surprise is a Python scikit building and analyzing recommender systems that deal with explicit rating data. Surprise stands for Simple Python Recommendation System Engine.

Surprise was designed with the following purposes in mind:

Give users perfect control over their experiments.
Alleviate the pain of dataset handling.
Provide various ready-to-use prediction algorithms.
Make it easy to implement new algorithm ideas.
Provide tools to evaluate, analyse and compare the algorithms performance.

For more info http://surpriselib.com

Performing Google Search using Python code

Install following two packages before executing the code below pip install beautifulsoup4 pip install google try: from googlesearch import search except ImportError: print("No module named 'google' found") # to search query = "python for engineers Blog" for j in search(query, tld="co.in", num=10, stop=1, pause=2): print(j)

Pycharm - The Python IDE for Professional Developers

Pycharm - Features

Intelligent Python Assistance
PyCharm provides smart code completion, code inspections, on-the-fly error highlighting and quick-fixes, along with automated code refactorings and rich navigation capabilities.

Web Development Frameworks
PyCharm offers great framework-specific support for modern web development frameworks such as Django, Flask, Google App Engine, Pyramid, and web2py.

Scientific Tools
PyCharm integrates with IPython Notebook, has an interactive Python console, and supports Anaconda as well as multiple scientific packages including matplotlib and NumPy.

Cross-technology Development
In addition to Python, PyCharm supports JavaScript, CoffeeScript, TypeScript, Cython, SQL, HTML/CSS, template languages, AngularJS, Node.js, and more.

Remote Development Capabilities
Run, debug, test, and deploy applications on remote hosts or virtual machines, with remote interpreters, an integrated ssh terminal, and Docker and Vagrant integration.

Built-in Developer Tools
A huge collection of tools out of the box: an integrated debugger and test runner; Python profiler; a built-in terminal; and integration with major VCS and built-in Database Tools.

Pylint - Python source code analyser

Pylint is a Python source code analyser which looks for programming errors, helps to enforce a coding standard, and other such. This quality checker for Python programming includes several features such as coding standard where it checks for the length of line codes, error detection, refactoring by detecting the duplicated code, Pylint is shipped with Pyreverse which creates UML diagrams for python code.

Courses in Artificial Intelligence - Study Options

• University of Edinburgh
• University of Amsterdam
• University of Georgia
• University of Groningen
• University of Southampton
• University of Hyderabad
• IIT Hyderabad
• IISc Bangalore
• Amrita Vishwa Vidyapeetham University

Walrus Operator in Python 3.8

One of the biggest highlights of Python 3.8.0 is a new feature for assignment expressions known as the Walrus Operator.

"There is new syntax := that assigns values to variables as part of a larger expression," the Python 3.8.0 release notes state.

"It is affectionately known as 'the walrus operator' due to its resemblance to the eyes and tusks of a walrus."

In this example, the assignment expression helps avoid calling len() twice:

if (n := len(a)) > 10:

    print(f"List is too long ({n} elements, expected <= 10)")

What’s New In Python 3.8

Download Python 3.8.0 Alpha 1

Microsoft: We want you to learn Python programming language for free

Microsoft has launched a new 44-part series called Python for Beginners on YouTube, consisting of three- to four-minute lessons from two self-described geeks at Microsoft who love programming and teaching in YouTube or  Channel9

The Python for Beginners series is presented by Christopher Harrison, a senior program manager at Microsoft, and Susan Ibach, a business development manager from Microsoft's AI Gaming unit. 

Microsoft has published a page on GitHub containing additional resources, including slides and code samples to help students become better at Python. 

The Developers have a option to interface with machine-learning frameworks like Google-developed TensorFlow, and the Microsoft Cognitive Toolkit (CNTK). 

DDA Line Drawing Algorithms Line Coordinates

def ROUND(a):
  return int(a + 0.5)
def drawDDA(x1,y1,x2,y2):
  x,y = x1,y1

  length = (x2-x1) if (x2-x1) > (y2-y1) else (y2-y1)
  dx = (x2-x1)/float(length)
  dy = (y2-y1)/float(length)
  print ('x = %s, y = %s' % (((ROUND(x),ROUND(y)))))
  for i in range(length):
    x += dx
    y += dy
    print ('x = %s, y = %s' % (((ROUND(x),ROUND(y)))))
drawDDA(2,5,10,20)

PyRobot - Python for Robotics

PyRobot is a Python package for benchmarking and running experiments in robot learning. The goal of this project is to abstract away the low-level controls for individual robots from the high-level motion generation and learning in an easy-to-use way. Using PyRobot will allow you to run robots without having to deal with the robot specific software along with enabling better comparisons.

pyrobot 0.1.0

Python developers reveal their favorite tool kits

• Python is used mainly for Data Analysis 
• NumPy is most popular data science framework 
• Flask is most popular web frameworks 
• Requests is most popular software libraries 
• PyCharm is most popular IDEs for Python 

More Details & Credits

Top 20 Job Sites for Python Programmers

1. Indeed.com
2. Python.org
3. Naukri.com
4. Remote Python
5. Python Jobs
6. Toptal 
7. Hired
8. GitHub 
9. Stack Overflow 
10. Upwork
11. Gun
12. Weworkremotely
13. Anaconda
14. X-Team
15. Authentic
16. Peopleperhour
17. Guru
18. Linkedin
19. Freelancer
20. Simplyhired

Python program to print memory and processor usage

import os import psutil pid = os.getpid() py = psutil.Process(pid) mu = (py.memory_info()[0] / 2.**30) * 1000 print('Memory Use(MB):', mu, 'of process id:', pid) print('CPU Use:',psutil.cpu_percent())

Python Program to Find the Fibonacci Series with defined program storage infinity

a=0 b=1 n=9999999999999999999999 # Number of terms in Fibonacci Series (sample value) # An integer giving the maximum value a variable of type Py_ssize_t can take. # It 's usually 2^31 - 1 on a 32-bit platform and 2^63 - 1 on a 64-bit platform. print(a,b,end=" ") while(n-2): c=a+b a=b b=c print("\n",c) n=n-1

Python Web Development Frameworks

Full Stack Frameworks

It gives full support to developers including basic components like form generators, form validation, and template layouts etc

1. Django is a high-level Python Web application development framework that encourages us to develop things rapidly, It uses pragmatic design.  It’s free and open source.

2. Web2py is a free open source full-stack development framework in python which allows the user to develop things quickly. It is a cross-platform framework that supports all popular operating systems. 

3. TurboGears is a free, open source and data-driven full-stack web application development Python framework. With the help of Javascript developer tools, developers can simply the web application.

4. CubicWeb is a semantic, free and open-source Python web framework, that empowers developers to efficiently build web applications by reusing components and following the well known object-oriented design principles. 

Non Full Stack Frameworks

Non-full stack frameworks are also called as Micro frameworks because it doesn’t have many components like full stack frameworks.

1. Flask is a microframework for Python based on Werkzeug, and Jinja 2. The main purpose is to develop a strong web application base. As compared to Django, Flask is best suited for small and easy projects.

2. CherryPy is a Minimalist Python Web Framework. It uses the Object-Oriented paradigm to develop web applications. This approach helps developers to develop web applications within a short period of time.

3. Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library. It is an easy-to-use lightweight framework generally used to build small web applications. It is mainly used to develop API’s.

4. Tornado is a python web framework with asynchronous network library. By using the non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.

Top 20 Python libraries

1. Requests. The most famous http library written by kenneth reitz.

2. Scrapy. If you are involved in webscraping then this is a must have library for you.

3. wxPython. A gui toolkit for python. I have primarily used it in place of tkinter.

4. Pillow. A friendly fork of PIL (Python Imaging Library).

5. SQLAlchemy. A database library.

6. BeautifulSoup. The xml and html parsing library is very useful for beginners.

7. Twisted. The most important tool for any network application developer.

8. NumPy.  It provides some advance math functionalities to python.

9. SciPy.  It is a library of algorithms and mathematical tools for python.

10. matplotlib. A numerical plotting library. It is very useful for any data scientist or any data analyzer.

11. Pygame.  This library will help you achieve your goal of 2d game development.

12. Pyglet. A 3d animation and game creation engine.

13. pyQT. A GUI toolkit for python.

14. pyGtk.  It is the same library in which the famous Bittorrent client is created.

15. Scapy. A packet sniffer and analyzer for python made in python.

16. pywin32. A python library which provides some useful methods and classes for interacting with windows.

17. nltk. Natural Language Toolkit

18. nose. A testing framework for python.

19. SymPy. SymPy can do algebraic evaluation, differentiation, expansion, complex numbers, etc.

20. IPython. It has completion, history, shell capabilities, and a lot more.

Python program to read a CSV file using Pandas

data.csv

import pandas as pd
X=pd.read_csv("data.csv")
type(X)
X
X.info( )
X.head( )
X.head(10)

Python program to renaming the file or directory

# X is the original name and Y is the name to be changed
import os, sys
print ("The dir is: %s"%os.listdir(os.getcwd()))
os.rename("X","Y")  
print ("Successfully renamed.")
print ("the dir is: %s" %os.listdir(os.getcwd()))

Python Libraries for General AI

• AIMA – Python implementation of algorithms from Russell and Norvig’s ‘Artificial Intelligence: A Modern Approach’
• pyDatalog – Logic Programming engine in Python
• SimpleAI – Python implementation of many of the artificial intelligence algorithms described on the book “Artificial Intelligence, a Modern Approach”. It focuses on providing an easy to use, well documented and tested library.
• EasyAI – Simple Python engine for two-players games with AI (Negamax, transposition tables, game solving).