Set environment variables for executing a command

Posted on

var1=foo1 var2=foo2 python run.py

Advertisements

Access Mysql Database inside Vagrant from Host Machine

Posted on Updated on

Step 1.

Edit your mysql config file

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Replace

bind-address = 127.0.0.1

with

bind-address = 0.0.0.0

0.0.0.0 implies the mysql server can be accessible from any ip. If you want to allow only certain ips, then replace 127.0.0.1 with the desired ip

Step 2.

Restart mysql server

sudo service mysql restart

Step 3.

By default your mysql user is set to accept connection from localhost. To access database on the mysql server from external host, the user’s hostname need to be set.

CREATE USER '[username]'@'%' IDENTIFIED BY '[password]';

Replace [username] and [password] from above with your desired username and password

Grant desired permission to the user

GRANT ALL ON *.* TO 'username'@'%';

Now you should be able to access the database from host machine with the username that you just set.

Thank  you for reading. Cheers!!!

Idiomatic Python

Posted on Updated on

Some handy python coding practices

Source: http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html

We often have to initialize dictionary entries before use:
This is the naïve way to do it:

navs = {}
for (foo, equity, position) in data:
    if foo not in navs:
        navs[foo] = 0
    navs[foo] += position * prices[equity]

Better way:

navs = {}
for (foo, equity, position) in data:
    navs[foo] = (navs.get(foo, 0) + position * prices[equity])

Initializing mutable dictionary values:

equities = {}
for (foo, equity) in data:
    if foo in equities:
        equities[foo].append(equity)
    else:
        equities[foo] = [equity]

dict.setdefault(key, default) does the job much more efficiently:

equities = {}
for (foo, equity) in data:
    equities.setdefault(foo, []).append(equity)

Even more better way would be

from collections import defaultdict
equities = defaultdict(list)
for (foo, equity) in data:
    equities[foo].append(equity)

Template strings

To deal with user-exposed format strings (due to security issues).

In:
from string import Template

t = Template('I am $name.')
t.substitute(name='Groot')
Out:
'I am Groot.'

Class vs Instance Variable Pitfalls

Class variables are shared across all instances of class; instance variables are unique to instance.

class Car:
    wheels = 4  # class variable
    car_instances = 0  # class variable (this one is actually useful)
    
    def __init__(self, color):
        self.color = color  # instance variable
        self.__class__.car_instances += 1  # note the extra __class__
        
a = Car('red')
b = Car('blue')

print('Cars existing:', Car.car_instances)

a.color = 'orange'
print(a.color, b.color)

a.wheels = 3  # creates a new variable a.wheels print(a.wheels, b.wheels, Car.wheels, a.__class__.wheels)
 Output:
Cars existing: 2
orange blue
3 4 4 4

Merging dicts

a = {'1': 1, '2': 2}
b = {'2': 3, '4': 4}
{**a, **b}

Output:

{‘1’: 1, ‘2’: 3, ‘4’: 4}

Crazy dict

dicts treat keys as identical if their hash values are the same and their __eq__ is True.

a = {True: 'yes', 1: 'no', 1.0: 'maybe'}

Output:

{True: ‘maybe’}

Different between tuple and list

Creating a list of 10 elements involves these steps

dis.dis('a = [1,2,3,4,5,6,7,8,9,10]')
 1 0 LOAD_CONST 0 (1)
 3 LOAD_CONST 1 (2)
 6 LOAD_CONST 2 (3)
 9 LOAD_CONST 3 (4)
 12 LOAD_CONST 4 (5)
 15 LOAD_CONST 5 (6)
 18 LOAD_CONST 6 (7)
 21 LOAD_CONST 7 (8)
 24 LOAD_CONST 8 (9)
 27 LOAD_CONST 9 (10)
 30 BUILD_LIST 10
 33 STORE_NAME 0 (a)
 36 LOAD_CONST 10 (None)
 39 RETURN_VALUE

Creating tuple involves these steps

dis.dis('a = (1,2,3,4,5,6,7,8,9,10)')
 1 0 LOAD_CONST 11 ((1, 2, 3, 4, 5, 6, 7, 8, 9, 10))
 3 STORE_NAME 0 (a)
 6 LOAD_CONST 10 (None)
 9 RETURN_VALUE

Use else to execute code after a for loop concludes

Harmful

for user in get_all_users():
    has_malformed_email_address = False
    print ('Checking {}'.format(user))
    for email_address in user.get_all_email_addresses():
        if email_is_malformed(email_address):
            has_malformed_email_address = True
            print ('Has a malformed email address!')
            break
    if not has_malformed_email_address:
        print ('All email addresses are valid!')

Idiomatic

for user in get_all_users():
    print ('Checking {}'.format(user))
    for email_address in user.get_all_email_addresses():
        if email_is_malformed(email_address):
            print ('Has a malformed email address!')
            break
    else:
        print ('All email addresses are valid!')

Avoid using ”, [], and {} as default parameters to functions

Harmful

def f(a, L=[]):
    L.append(a)
    return L
print(f(1))
print(f(2))
print(f(3))
# This will print
#
# [1]
# [1, 2]
# [1, 2, 3]

Idiomatic

def f(a, L=None):
    if L is None:
        L = []
    L.append(a)
    return L
print(f(1))
print(f(2))
print(f(3))
# This will print
# [1]
# [2]
# [3]

Use the * operator to represent the “rest” of a list

Harmful

some_list = ['a', 'b', 'c', 'd', 'e']
(first, second, rest) = some_list[0], some_list[1], some_list[2:]
(first, middle, last) = some_list[0], some_list[1:-1], some_list[-1]
(head, penultimate, last) = some_list[:-2], some_list[-2], some_list[-1]

Idiomatic

some_list = ['a', 'b', 'c', 'd', 'e']
(first, second, *rest) = some_list
(first, *middle, last) = some_list
(*head, penultimate, last) = some_list

Prefer a generator expression to a list comprehension for simple iteration

A list comprehension generates a list object and fills in all of the elements immediately. For large lists, this can be prohibitively expensive. The generator returned by a generator expression, on the other hand, generates each element “on-demand”.

Harmful:

for uppercase_name in [name.upper() for name in get_all_usernames()]:
    process_normalized_username(uppercase_name)

Idiomatic:

for uppercase_name in (name.upper() for name in get_all_usernames()):
    process_normalized_username(uppercase_name)

Shorten the try, catch, pass occurring pattern

We do this most of the time when we don’t care about the exception

try:
    foo()
except SomeException:
    pass

Do this instead

from contextlib import suppress
with suppress(SomeException):
    os.remove('i_probably_do_not_exist')

Gradle Update Project Version number through gradle task

Posted on

Create a gradle.properties file and add a version property

version=1.0.0-SNAPSHOT

Then in build.gradle file add

task patchVersion {
    doLast{
        Properties props = new Properties()
        File propsFile = new File('gradle.properties')
        props.load(propsFile.newDataInputStream())
        def stable_project_version = props.getProperty('version') as String
        if (stable_project_version.indexOf('-SNAPSHOT') > 0){
            throw new GradleException('Cannot patch a non SNAPSHOT version')
        } else {
            def (major, minor, patch) = stable_project_version.tokenize('.')
            patch = String.valueOf(patch.toInteger() + 1)
            String snapshot_project_version = major + '.' + minor + '.' + patch + '-SNAPSHOT'
            props.setProperty('version', snapshot_project_version)
            props.store(propsFile.newWriter(), null)
            println(snapshot_project_version)
        }
    }
}

task releaseVersion {
    doLast{
        Properties props = new Properties()
        File propsFile = new File('gradle.properties')
        props.load(propsFile.newDataInputStream())
        def snapshot_project_version = props.getProperty('version') as String
        if (snapshot_project_version.indexOf('-SNAPSHOT') > 0){
            def stable_project_version = snapshot_project_version - '-SNAPSHOT'
            props.setProperty('version', stable_project_version)
            props.store(propsFile.newWriter(), null)
            println(stable_project_version)
        } else {
            throw new GradleException('Cannot release a non SNAPSHOT version')
        }
    }
}

Run releaseVersion to automatically update SNAPSHOT version to stable version.
Eg. if your project version is 1.0.0-SNAPSHOT

gradle releaseVersion

will update the version in gradle.properties to 1.0.0

Run patchVersion to automatically update Stable version to the next patch version.
Eg. if your version is 1.0.0

gradle patchVersion

will update the version in gradle.properties to 1.0.1-SNAPSHOT

JSON to csv string in python

Posted on Updated on

from cStringIO import StringIO
import csv


def json_to_csv(data):
    output = StringIO()
    writer = csv.DictWriter(output, data[0].keys())
    writer.writeheader()
    for row in data:
        writer.writerow(row)
    return output


print (write_csv([{"name": "Dolor", "age": "23"}, {"name": "John Doe", "age": "32"}, {"age": "42", "name": "Lorem Ipsum"}], 'output.csv').getvalue())

# output
# age,name
# 23,Abiral
# 32,Lorem
# 42,asdfasd

AWS VPC and VPC Peering

Posted on Updated on

 

Creating a VPC

  1. Create a new VPC with CIDR block
    Eg .
    10.0.0.0/16
    Number of ips = 2 ^ (32-16)
    Ips range in the VPC (1.0.0.0 – 10.0.255.255)
  2. Create Public and private Subnets inside the VPC
    Eg. CIDR block (10.0.0.0/24) – 256 ips in a subnet
    Public subnet will have Internet gateway attached to their route table while private ones won’t
  3. Create an Internet Gateway and attach it to the VPC, Instance in the VPC will be able to connect to the internet though the Internet gateway
  4. Create a NAT Gateway inside a public subnet, NAT gateway allows one way traffic from Private Subnet to the internet
  5. Create 2 Route tables one for public subnet and one for private subnet
    Assign subnet to corresponding route table through subnet Association Tab

    1. For Private Subnet, add these routes
      1. For outgoing connection-
        Destination: 0.0.0.0/0 —- Target: NAT-gateway(eg. nat-foobar)
        Now instance in private subnet can access the outside world through NAT but outside world can’t connect to the instances
    2. For Public Sunet, add these routes
      1. For in and out connection through internet
        Destination: 0.0.0.0/0 —- Target: internet-gateway(eg. igw-loremipsum)
        Now instances in public subnet can access and be accessed from the internet through the Internet Gateway

VPC Peering

  1. Make sure the ip CIDR block of the 2 VPCS do not collide. If the existing VPC has CIDR block 10.0.0.0/16, then the new VPC should use CIDR block like 10.1.0.0/16 so the ips in the two VPCs don’t collide
  2. Creating Peering Connection from Peering Connections Page
    1. Give a proper Peering connection name tag
    2. Select new VPC as the Requester VPC
    3. Select old VPC as the another VPC and submit
    4. Click on the Newly Created Peering Connection and press Actions and press Accept Request to accept the peering.
  3. Now the route tables need to be updated to add routes to each others ips
    1. Edit the private and public Route table of both VPC and add an entry on each
      Destination = CIDR block of the other VPC
      Target = Peering connection. Eg pcx-foobar
    2. Edit DNS Settings:
      Check both checkboxes
      Allow DNS resolution from peer VPC (vpc-dc700db8) to private IP
      Allow DNS resolution from local VPC (vpc-6bcf1b0f) to private IP
  4. For accessing rds from other VPC, add the CIDR block of the current VPC in the security group of rds instance.

Find max and min number from a number having the same number of digits as that number

Posted on Updated on

foo_number = 12345
min = pow(10, len(str(foo_number)) - 1) # 10000
max = pow(10, len(str(foo_number))) - 1 # 99999