GENERAL | https://github.com/ml874/Data-Engineering-on-GCP-Cheatsheet | GCP Cheatsheet (data_engineering_on_GCP.pdf) |
GENERAL | https://medium.com/@hello_92179 | GCP cheat sheets by Yaron Hollander |
GCE | https://cloud.google.com/compute/ | Compute Engine – Scalable, High-Performance Virtual Machines |
GCE | https://cloud.google.com/compute/docs/instances/preemptible | Preemptible VM Instances |
GCE | https://cloud.google.com/compute/docs/disks/ | Storage Options |
GCE | https://cloud.google.com/compute/docs/instance-groups/ | Instance Groups |
GCE | https://cloud.google.com/compute/docs/startupscript | Running Startup Scripts |
GCE | https://cloud.google.com/compute/docs/shutdownscript | Running Shutdown Scripts |
GCE | https://cloud.google.com/compute/docs/sustained-use-discounts | Sustained Use Discounts |
Networking | https://cloud.google.com/vpc/docs/overview | Overview of Virtual Private Cloud |
Networking | https://cloud.google.com/vpc/docs/vpc | Virtual Private Cloud (VPC) Network Overview |
Networking | https://cloud.google.com/vpc/docs/advanced-vpc | Advanced VPC Concepts |
Networking | https://cloud.google.com/vpc/docs/shared-vpc | Shared VPC Overview |
Networking | https://cloud.google.com/vpc/docs/vpc-peering | VPC Network Peering |
Networking | https://cloud.google.com/vpc/docs/private-access-options | Private Access Options for Services |
Networking | https://cloud.google.com/vpn/docs/concepts/overview | Cloud VPN Overview |
Networking | https://cloud.google.com/interconnect/docs/concepts/overview | Cloud Interconnect Overview |
Networking | https://cloud.google.com/interconnect/docs/how-to/choose-type | Choose an interconnect type |
Networking | https://cloud.google.com/nat/docs/overview | Cloud NAT |
GCS | https://cloud.google.com/appengine/docs/standard/java/googlecloudstorageclient/understanding-storage-features | Understanding Google Cloud Storage Features |
GCS | https://cloud.google.com/storage/docs/key-terms | GCS Key Terms |
GCS | https://cloud.google.com/storage/docs/storage-classes | Storage Classes |
GCS | https://cloud.google.com/storage/docs/bucket-locations | Bucket Locations |
GCS | https://cloud.google.com/storage/docs/encryption/ | Data encryption options |
GCS | https://cloud.google.com/storage/docs/metadata | Object metadata |
GCS | https://cloud.google.com/storage/docs/pubsub-notifications | Cloud Pub/Sub notifications for Cloud Storage |
GCS | https://cloud.google.com/storage/docs/object-change-notification | Object change notification |
GCS | https://cloud.google.com/storage/docs/object-versioning | Object Versioning |
GCS | https://cloud.google.com/storage/docs/lifecycle | Object Lifecycle Management |
GCS | https://cloud.google.com/storage/docs/requester-pays | Requester Pays |
GCS | https://cloud.google.com/storage/docs/bucket-lock | Retention policies using Bucket Lock |
GCS | https://cloud.google.com/storage/docs/transcoding | Transcoding of gzip-compressed files |
GCS | https://cloud.google.com/storage/docs/cross-origin | Cross-origin resource sharing (CORS) |
GCS | https://cloud.google.com/storage/docs/collaboration | Sharing and Collaboration |
GCS | https://cloud.google.com/storage/docs/access-control/ | Access control options |
GCS | https://cloud.google.com/storage/docs/access-control/iam | Cloud Identity and Access Management |
GCS | https://cloud.google.com/storage/docs/access-control/lists | Access Control Lists (ACLs) |
GCS | https://cloud.google.com/storage/docs/access-control/cookie-based-authentication | Performing authenticated browser downloads |
GCS | https://cloud.google.com/storage/docs/access-control/signed-urls | Signed URLs |
GCS | https://cloud.google.com/storage/docs/xml-api/post-object | POST Object |
GCS | https://cloud.google.com/storage/docs/composite-objects | Composite Objects and Parallel Uploads |
GCS | https://cloud.google.com/storage-transfer/docs/overview | Storage Transfer Service Overview |
GCS | https://cloud.google.com/storage/docs/best-practices | Best practices for Cloud Storage |
Cloud SQL | https://cloud.google.com/sql/ | Cloud SQL |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/high-availability | Overview of the High Availability Configuration |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/external-connection-methods | Connection options for external applications |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/sql-proxy | About the Cloud SQL Proxy |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/replication/ | Replication options |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/import-export/ | Best practices for importing and exporting data |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/backup-recovery/backups | Overview of Backups |
Cloud SQL | https://cloud.google.com/sql/docs/mysql/backup-recovery/restore | Overview of Restoring an Instance |
Cloud SQL | https://cloud.google.com/sql/faq | Cloud SQL FAQ |
BIGQUERY | https://cloud.google.com/bigquery/what-is-bigquery | What is BigQuery? |
BIGQUERY | https://cloud.google.com/bigquery/docs/projects | Projects |
BIGQUERY | https://cloud.google.com/bigquery/docs/storing-data | Storing data |
BIGQUERY | https://cloud.google.com/bigquery/docs/loading-data | Introduction to loading data into BigQuery |
BIGQUERY | https://cloud.google.com/bigquery/streaming-data-into-bigquery | Streaming Data into BigQuery |
BIGQUERY | https://cloud.google.com/bigquery/docs/loading-data-local | Loading data from a local data source |
BIGQUERY | https://cloud.google.com/bigquery/docs/loading-data-cloud-storage | Introduction to loading data from Cloud Storage |
BIGQUERY | https://cloud.google.com/bigquery/docs/partitioned-tables | Partitioned Tables |
BIGQUERY | https://cloud.google.com/bigquery/docs/clustered-tables | Introduction to clustered tables |
BIGQUERY | https://cloud.google.com/bigquery/pricing#queries | Query pricing |
BIGQUERY | https://cloud.google.com/bigquery/docs/exporting-data | Exporting table data |
BIGQUERY | https://cloud.google.com/bigquery/external-data-sources | Introduction to external data sources |
BIGQUERY | https://cloud.google.com/bigquery/docs/access-control | Access control |
BIGQUERY | https://cloud.google.com/bigquery/docs/dataset-access-controls | Controlling access to datasets |
BIGQUERY | https://cloud.google.com/bigquery/docs/scan-with-dlp | Using Cloud DLP to scan BigQuery data |
BIGQUERY | https://cloud.google.com/bigquery/docs/customer-managed-encryption | Protecting data with Cloud KMS keys |
BIGQUERY | https://cloud.google.com/bigquery/docs/slots | Slots |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-costs | BigQuery best practices: Controlling costs |
BIGQUERY | https://cloud.google.com/bigquery/docs/updating-datasets | Updating dataset properties |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-storage | BigQuery Best practices: Optimizing storage |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-performance-input | Managing input data and data sources |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-performance-communication | Optimizing communication between slots |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-performance-compute | Optimizing query computation |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-performance-output | Managing query outputs |
BIGQUERY | https://cloud.google.com/bigquery/docs/best-practices-performance-patterns | Avoiding SQL anti-patterns |
BIGQUERY | https://cloud.google.com/bigquery/query-plan-explanation | Query plan and timeline |
BIGQUERY | https://cloud.google.com/bigquery/docs/views-intro | Introduction to views |
BIGQUERY | https://cloud.google.com/bigquery/docs/view-access-controls | Controlling access to views |
BIGQUERY | https://cloud.google.com/bigquery/docs/authorized-views | Creating authorized views |
BIGQUERY | https://cloud.google.com/bigquery/docs/transfer-service-overview | Introduction to BigQuery Data Transfer Service |
BIGQUERY | https://cloud.google.com/bigquery/sql-reference/data-manipulation-language | Data Manipulation Language |
BIGQUERY | https://cloud.google.com/bigquery/docs/cached-results | Using cached query results |
BIGQUERY | https://cloud.google.com/bigquery/docs/querying-wildcard-tables | Querying multiple tables using a wildcard table |
BIGTABLE | https://cloud.google.com/bigtable/docs/overview | Overview of Cloud Bigtable |
BIGTABLE | https://cloud.google.com/bigtable/docs/instances-clusters-nodes | Instances, clusters, and nodes |
BIGTABLE | https://cloud.google.com/bigtable/docs/schema-design | Designing Your Schema |
BIGTABLE | https://cloud.google.com/bigtable/docs/schema-design-time-series | Schema Design for Time Series Data |
BIGTABLE | https://cloud.google.com/bigtable/docs/access-control | Access Control |
BIGTABLE | https://cloud.google.com/bigtable/docs/replication-overview | Overview of Replication |
BIGTABLE | https://cloud.google.com/bigtable/docs/replication-settings | Examples of Replication Settings |
BIGTABLE | https://cloud.google.com/bigtable/docs/app-profiles | Application Profiles |
BIGTABLE | https://cloud.google.com/bigtable/docs/failovers | Failovers |
BIGTABLE | https://cloud.google.com/bigtable/docs/keyvis-overview | Overview of Key Visualizer |
BIGTABLE | https://cloud.google.com/bigtable/docs/keyvis-patterns | Heatmap Patterns |
BIGTABLE | https://cloud.google.com/bigtable/docs/performance | Understanding Cloud Bigtable performance |
BIGTABLE | https://cloud.google.com/bigtable/docs/choosing-ssd-hdd | Choosing Between SSD and HDD Storage |
DB COMPARISON | http://stackoverflow.com/questions/30085326/google-cloud-bigtable-vs-google-cloud-datastore | Google Cloud Bigtable vs Google Cloud Datastore |
DB COMPARISON | https://db-engines.com/en/system/Google+Cloud+Bigtable%3BGoogle+Cloud+Datastore | System Properties Comparison Google Cloud Bigtable vs. Google Cloud Datastore |
DB COMPARISON | http://terrenceryan.com/blog/index.php/when-to-pick-google-bigtable-vs-other-cloud-platform-databases/ | When to Pick Google Bigtable vs Other Cloud Platform Databases |
CLOUD DATAFLOW | https://cloud.google.com/dataflow/docs/concepts/beam-programming-model | Programming model for Apache Beam |
DATAFLOW | https://beam.apache.org/documentation/programming-guide/ | Apache Beam Programming Guide |
DATAFLOW | https://cloud.google.com/dataflow/docs/concepts/security-and-permissions | Cloud Dataflow security and permissions |
DATAFLOW | https://cloud.google.com/dataflow/docs/concepts/access-control | Cloud Dataflow Access Control Guide |
DATAFLOW | https://stackoverflow.com/questions/33518104/google-dataflow-vs-apache-spark | Google Dataflow vs Apache Spark |
DATAFLOW STACKDRIVER | https://cloud.google.com/blog/big-data/2017/03/monitoring-and-improving-your-google-cloud-dataflow-pipelines-with-google-stackdriver.html | Monitoring and improving your Google Cloud Dataflow pipelines with Google Stackdriver |
DATAPROC | https://cloud.google.com/dataproc/docs/concepts/overview | What is Google Cloud Dataproc? |
DATAPROC | https://cloud.google.com/dataproc/docs/concepts/compute/custom-machine-types | Custom machine types |
DATAPROC | https://cloud.google.com/dataproc/docs/concepts/compute/dataproc-local-ssds | Cloud Dataproc Local SSDs |
DATAPROC | https://cloud.google.com/dataproc/docs/concepts/preemptible-vms | Preemptible VMs |
DATAPROC | https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/init-actions | Initialization actions |
DATAPROC | https://cloud.google.com/dataproc/docs/resources/faq | Cloud Dataproc FAQ |
DATAPROC | https://cloud.google.com/dataproc/docs/connectors/cloud-bigtable | Google Cloud Bigtable and Cloud Dataproc |
DATAPROC | https://cloud.google.com/dataproc/docs/connectors/cloud-storage | Google Cloud Storage connector |
DATAPROC | https://www.quora.com/What-is-the-criteria-to-chose-Pig-Hive-Hbase-Storm-Solr-or-Spark-to-analyze-your-data-in-Hadoop | What is the criteria to chose Pig, Hive, Hbase, Storm, Solr, or Spark to analyze your data in Hadoop? |
DATAPROC | https://stackoverflow.com/questions/13911501/when-to-use-hadoop-hbase-hive-and-pig | When to use Hadoop, HBase, Hive and Pig? |
DATAPROC | https://community.cloudera.com/t5/Hadoop-101-Training-Quickstart/The-Differences-between-Pig-Hive-and-HBase/ta-p/31007 | The Differences between Pig, Hive, and HBase |
DATAPROC | https://www.dezyre.com/article/difference-between-pig-and-hive-the-two-key-components-of-hadoop-ecosystem/79 | Difference between Pig and Hive-The Two Key Components of Hadoop Ecosystem |
DATAPROC | http://www.hadoop360.datasciencecentral.com/blog/pig-vs-hive-vs-sql-difference-between-the-big-data-tools?context=tag-hadoop | Pig vs Hive vs SQL – Difference between the Big Data Tools |
DATAPROC | https://hadoopecosystemtable.github.io/ | The Hadoop Ecosystem Table |
DATAPROC | https://www.smartdatacollective.com/hadoop-toolbox-when-use-what/ | Hadoop Toolbox: When to Use What |
STACKDRIVER MONITORING | https://cloud.google.com/monitoring/docs/ | Stackdriver Monitoring documentation |
STACKDRIVER LOGGING | https://cloud.google.com/logging/docs/basic-concepts | Basic Concepts |
STACKDRIVER LOGGING | https://cloud.google.com/logging/docs/export/using_exported_logs | Using Exported Logs |
STACKDRIVER LOGGING | https://cloud.google.com/logging/docs/agent/ | The Logging Agent |
STACKDRIVER LOGGING | https://cloud.google.com/logging/docs/audit/ | Cloud Audit Logging |
Stackdriver Debugger | https://cloud.google.com/debugger/docs/ | Stackdriver Debugger |
STORAGE | https://cloud.google.com/storage-options/ | Choosing a Storage Option |
CLOUD PUB/SUB | https://cloud.google.com/pubsub/docs/overview | What is Google Cloud Pub/Sub? |
CLOUD PUB/SUB | https://cloud.google.com/pubsub/docs/tasks-vs-pubsub | Choosing between Cloud Tasks and Cloud Pub/Sub |
CLOUD PUB/SUB | https://cloud.google.com/pubsub/docs/subscriber | Subscriber overview |
CLOUD PUB/SUB | https://cloud.google.com/pubsub/docs/publisher | Publishing messages |
CLOUD PUB/SUB | https://cloud.google.com/pubsub/docs/ordering | Ordering messages |
CLOUD PUB/SUB | https://cloud.google.com/pubsub/docs/replay-overview | Replaying and discarding messages |
IAM | https://cloud.google.com/iam/docs/overview | IAM Overview |
IAM | https://cloud.google.com/compute/docs/access/service-accounts | Service Accounts |
IAM | https://cloud.google.com/iam/docs/resource-hierarchy-access-control | Using Resource Hierarchy for Access Control |
IAM | https://cloud.google.com/iam/docs/using-iam-securely | Using IAM Securely |
IAM | https://cloud.google.com/resource-manager/docs/cloud-platform-resource-hierarchy | Resource Hierarchy |
Security | https://cloud.google.com/armor/docs/security-policy-concepts | Google Cloud Armor Security Policy Concepts |
Security | https://cloud.google.com/kms/docs/hsm | Cloud HSM |
Security | https://cloud.google.com/kms/ | CLOUD KEY MANAGEMENT SERVICE |
Security | https://cloud.google.com/dlp/ | Cloud Data Loss Prevention |
Security | https://cloud.google.com/vpc-service-controls/docs/overview | Overview of VPC Service Controls |
Security | https://cloud.google.com/security-command-center/docs/concepts-overview | Cloud SCC conceptual overview |
Security | https://cloud.google.com/security-scanner/ | CLOUD SECURITY SCANNER |
Security | https://cloud.google.com/identity/docs/concepts/overview | Google Identity Platform Overview |
Security | https://cloud.google.com/iap/docs/concepts-overview | Cloud Identity-Aware Proxy overview |
security and compliance | https://cloud.google.com/security/gdpr/ | Google Cloud & the General Data Protection Regulation (GDPR) |
security and compliance | https://cloud.google.com/security/compliance/pci-dss/ | Standards, Regulations & Certifications |
security and compliance | https://cloud.google.com/solutions/pci-dss-compliance-in-gcp | PCI Data Security Standard compliance |
Security | https://cloud.google.com/blog/products/gcp/managing-encryption-keys-in-the-cloud-introducing-google-cloud-key-management-service?m=1 | Managing encryption keys in the cloud: introducing Google Cloud Key Management Service |
Security | https://cloud.google.com/security/encryption-at-rest/ | ENCRYPTION AT REST |
Security | https://cloud.google.com/security/compliance/hipaa-compliance/ | HIPAA |
Security | https://cloud.google.com/security/compliance/coppa/ | COPPA (U.S.) |
Security | https://cloud.google.com/security/compliance/fedramp/ | FedRAMP |
| | |
MACHINE LEARNING ENGINE | https://cloud.google.com/ml-engine/ | CLOUD MACHINE LEARNING ENGINE |
Storage | https://cloud.google.com/filestore/ | CLOUD FILESTORE |
Storage | https://cloud.google.com/products/data-transfer/ | CLOUD DATA TRANSFER |
Cloud Composer | https://cloud.google.com/composer/docs/concepts/overview | Overview of Cloud Composer |
Transfer Appliance | https://cloud.google.com/transfer-appliance/docs/2.0/overview | Transfer Appliance Overview |
IOT | https://cloud.google.com/iot/docs/concepts/overview | Cloud IoT Core overview |
Cloud Spanner | https://cloud.google.com/spanner/docs/overview | Cloud Spanner |
Cloud Spanner | https://cloud.google.com/spanner/docs/schema-and-data-model | Schema and data model |
Cloud Spanner | https://cloud.google.com/spanner/docs/schema-design | Schema design best practices |
MACHINE LEARNING | https://cloud.google.com/automl/ | Cloud AutoML |
MACHINE LEARNING | https://cloud.google.com/automl-tables/ | AutoML Tables |
MACHINE LEARNING | https://cloud.google.com/deep-learning-vm/ | Deep Learning VM Image |
MACHINE LEARNING | https://cloud.google.com/tpu/docs/tpus | Cloud Tensor Processing Units (TPUs) |
MACHINE LEARNING | https://opensource.com/article/18/12/introduction-kubeflow | An introduction to Kubeflow |
| | |
MLENGINE / TENSORFLOW | https://cloud.google.com/ml-engine/docs/tensorflow/projects-models-versions-jobs | Projects, Models, Versions, and Jobs |
MLENGINE / TENSORFLOW | https://cloud.google.com/ml-engine/docs/tensorflow/ml-solutions-overview | Machine Learning Workflow |
MLENGINE / TENSORFLOW | https://cloud.google.com/ml-engine/docs/tensorflow/data-prep | Preparing Data |
MLENGINE / TENSORFLOW | https://cloud.google.com/ml-engine/docs/tensorflow/training-overview | Training Overview |
MLENGINE / TENSORFLOW | https://cloud.google.com/ml-engine/docs/tensorflow/hyperparameter-tuning-overview | Overview of Hyperparameter Tuning |
MLENGINE / TENSORFLOW | https://cloud.google.com/ml-engine/docs/tensorflow/prediction-overview | Prediction Overview |
MACHINE LEARNING | https://cloud.google.com/ml-engine/docs/how-tos/managing-models-jobs | Managing Models and Jobs |
MACHINE LEARNING | http://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/ | Supervised and Unsupervised Machine Learning Algorithms |
DATALAB | https://cloud.google.com/datalab/ | CLOUD DATALAB |
DATALAB | https://cloud.google.com/blog/big-data/2016/05/how-to-forecast-demand-with-google-bigquery-public-datasets-and-tensorflow | How to forecast demand with Google BigQuery, public datasets and TensorFlow |
MACHINE LEARNING | https://www.tensorflow.org/get_started/summaries_and_tensorboard | ML – Tensorboard |
MACHINE LEARNING | https://www.tensorflow.org/get_started/graph_viz | ML – Learning visualization |
MACHINE LEARNING | https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-algorithm-choice | How to choose algorithms for Microsoft Azure Machine Learning |
MACHINE LEARNING | https://medium.com/@nomadic_mind/new-to-machine-learning-avoid-these-three-mistakes-73258b3848a4 | New to Machine Learning? Avoid these three mistakes |
kafka | https://kafka.apache.org/intro | Kafka – Introduction |
DATAPREP | https://cloud.google.com/dataprep/docs/concepts | Dataprep – Concepts |
Cloud Memorystore | https://cloud.google.com/memorystore/docs/redis/redis-overview | Overview of Cloud Memorystore for Redis |
Cloud Datastore | https://cloud.google.com/datastore/docs/concepts/overview | Cloud Datastore Overview |
Cloud Datastore | https://cloud.google.com/datastore/docs/firestore-or-datastore | Choosing between Native Mode and Datastore Mode |