Skip to main content

Posts

Clone/Copy Existing Code to New Code Repo in Git

How to Clone/Copy Existing Code to New Code Repo in Git? Clone Existing repo to new directory                 $git clone git@github.com:org/old-repo.git C:\GitHub\code-cov-test  go to code-cov-test dir                $cd ../code-cov-test Remove old repo git                 $rm -rf .git #if this command doesn't work go to directory and remove .git file which is hidden Do git initialization                $git init  Add all files to git                $git add . Commit the local changes                 $git commit -m "Initial Commit" Add new repo as remote if repo not present create new repo                $git remote add origin git@github.com:org/co...
Post a Comment
Read more
Recent posts

Python : Setup to use local Nexus repo to download packages

 If you want to use your own or nexus repo to download the packages , use below process Create folder if not exists in C:\Users\username\AppData\Roaming\pip in windows Create pip.ini file on windows AppData>Roaming>pip folder  Copy the below lines and place in pip.ini file                [global]               index-url=https://<repo.com>/nexus/repository/ae-pypi-group/simple               trusted-host=<repo.com> On unix default configuration file is $HOME/.config/pip/pip.conf which respects XDG_CONFIG_HOME environment variable. On MacOs the configuration file is $HOME/Library/Application Support/pip/pip.conf
Post a Comment
Read more

BQ Commands

HTML Table Generator Command Description Comments gcloud config unset auth/impoersonate_service_account  Unset Impersonate SA    gcloud config set auth/impersonate_service_account sa   Set Impersonate SA    gcloud config list  Gcloud Conifgutation list    bq query --project_id={project_id} --format=csv --use_legacy_sql=false 'select * from tbl'   Run SQL Query   
Post a Comment
Read more

Creating New Conda Environments

 Creating New Conda Environments.. Create New Environment Activate Virtual Environment Install Packages if any Install Jupyter Notebook if needed to add this new environment as a kernel into your Jupyter Notebook. $conda create -n env-name python=3.8 numpy pandas -c https://repo.anaconda.com/pkgs/snowflake $conda activate env-name $conda install snowflake-snowpack-python $conda activate env-name pip install ipykernel python -m ipykernel install --name env-name
Post a Comment
Read more

Port and Connections

Connection ID Connection Type Host Port airflow_db mysql airflow-sqlproxy-service.default.svc.cluster.local 3306 beeline_default beeline localhost 10000 cassandra_default cassandra cassandra 9042 druid_broker_default druid druid-broker 8082 druid_ingest_default druid druid-overlord 8081 hiveserver2_default hiveserver2 localhost 10000 http_default http https://www.httpbin.org metastore_default hive_metastore localhost 9083 mongo_default mongo mongo 27017 mysql_default mysql mysql 1433 presto_default presto localhost 3400 redis_default redis redis 6379 sftp_default sftp localhost 22 vertica_default vertica localh...
Post a Comment
Read more

Durability,Scalable,Availability

Availability,durability and scalability are design characteristics you need to consider when developing a cloud application.  Durability ensures that your data is protected and available. Snapshots are a viable way of backing your data. Durability is a measurement related to data protection. Autoscaling is a method for implementing scalable applications. Health checks and Managed Instance groups improve availability. Availability is a measurement of uptime
Post a Comment
Read more

CICD

 Continuous integration/continuous delivery (CI/CD) pipelines automate the testing and delivery of code by monitoring a controlled software repository. When new software is checked in, the pipeline orchestrator will first run unit tests. If the tests are successful , a deployment package is built and saved to a Container Registry,which completes the steps for continuous integration.Continuous delivery deploys your images or artifacts to the operational environment you specify, such as App Engine or GKE. With Google Cloud, you can use Cloud Source Repositories as your Version Control repository. Unit testing and container artifacts can be produced through Cloud Build and automated through build triggers. Common places to save and manage your finalized images include Container Registry and Artifact Registry. Continuous Delivery can also be implemented through steps specified in Cloud Build and implementing build triggers to apply build operations when a new manifest is added to a mon...
Post a Comment
Read more