How to List S3 Buckets and Objects Using AWS CLI

Amazon Simple Storage Service (S3) is a scalable cloud storage solution provided by AWS, widely used for storing data of all kinds. Whether you are managing backups, application files, or large datasets, the AWS CLI (Command Line Interface) is an essential tool for quickly interacting with S3. One of the most frequent tasks is listing buckets and objects in your S3 storage.

In this article, we’ll guide you through various methods of listing your S3 buckets and their contents using AWS CLI. We will explain each command and provide examples to help you get started quickly.

Prerequisites

• AWS CLI is installed: You can install it from the AWS CLI installation guide.
• AWS CLI is configured: Run the command aws configure to set up your credentials (Access Key, Secret Access Key, Region, etc.).
• Necessary permissions: Make sure your IAM user has the right permissions to access and list S3 buckets. The required permission is s3:ListBucket.

1. Listing All S3 Buckets

To list all the S3 buckets in your AWS account, use the following command:

aws s3 ls

This command will return a list of all S3 buckets with their creation dates.

Example Output:

2023-10-12 12:34:56 bucket-name-1
2023-09-10 08:21:33 bucket-name-2

2. Listing Contents of a Specific S3 Bucket

If you want to list all the objects in a specific bucket, you can append the bucket name to the command:

aws s3 ls s3://bucket-name

Replace bucket-name with the actual name of your S3 bucket.

Example Output:

2024-01-10 14:20:15    1024 file1.txt
2024-01-10 14:30:25    2048 file2.txt

3. Listing Objects in a Specific Folder

S3 buckets can contain virtual directories (folders). To list the contents of a specific folder within a bucket, specify the folder name:

aws s3 ls s3://bucket-name/folder-name/

Example Output:

2024-02-15 15:10:05    512  folder-name/file3.jpg
2024-02-16 10:12:45   1024  folder-name/file4.pdf

4. Listing Objects Recursively

To list all objects in a bucket, including those stored in subdirectories, use the --recursive option:

aws s3 ls s3://bucket-name --recursive

Example Output:

2024-01-10 14:20:15    1024 folder1/file1.txt
2024-01-10 14:30:25    2048 folder2/file2.txt
2024-01-11 09:15:10    512  folder2/subfolder/file3.jpg

5. Listing with Human-Readable File Sizes

To view file sizes in a human-readable format (e.g., KB, MB, GB), use the --human-readable option:

aws s3 ls s3://bucket-name --human-readable

Example Output:

2024-01-10 14:20:15   1.0 KiB folder1/file1.txt
2024-01-10 14:30:25   2.0 KiB folder2/file2.txt

6. Summarizing Total Files and Sizes

To get a summary of the total number of objects and their cumulative size in a bucket, use the --summarize option along with --recursive:

aws s3 ls s3://bucket-name --recursive --summarize

Example Output:

2024-01-10 14:20:15    1024 folder1/file1.txt
2024-01-10 14:30:25    2048 folder2/file2.txt

Total Objects: 2
Total Size: 3 KiB

7. Filtering Results Using Wildcards

You can filter the objects by file name patterns using wildcards:

aws s3 ls s3://bucket-name --recursive --exclude "*" --include "*.txt"

This command will only list .txt files, excluding other file types.

Common Errors and How to Fix Them

• Access Denied Error: Ensure that your IAM user has the necessary permissions to list the bucket contents. You need s3:ListBucket and possibly other permissions for more advanced actions.
• No Such Bucket: Verify that the bucket name is correct and exists in the region you’re working in.
• CLI Configuration Issues: Ensure the AWS CLI is properly configured using aws configure, and check if you’re using the correct AWS profile if necessary.

Using the AWS CLI to list S3 buckets and objects is a powerful way to interact with your storage without needing to navigate the AWS Management Console. Whether you're listing all buckets, viewing files in a folder, or summarizing the total size of a bucket, these commands provide flexibility and control over your cloud storage operations.

By mastering these CLI commands, you can streamline your cloud management processes and handle S3 tasks more efficiently, saving both time and effort.

Java Function Code to create Reverse Linked List of Singly Linked List (In memory pointer changes)

Here is a sample java Function Code to create Reverse Linked List of Singly Linked List (In memory pointer changes). Approach

• 1) Maintain a previous node pointer
• 2) maintain a current node pointer
• 3) store current-> next in temporary variable
• 4) replace the current node-> next to previous pointer
• 5) do step 3 and 4 till you reach current node is your end node
• 6) at the end update the head->next to null as the old linked list head will be your end node now
• 7) return previous node as this will be the head of your new linked list.

You can see the image below - before and after structure of linked list:

package com.sample.linkedlist;

public class ReverseLinkedList {

 static class Node {
  int data;
  Node next;

  public Node(int data) {
   this.data = data;
  }

  @Override
  public String toString() {
   return "Node[data=" + data + "]";
  }

 }

 public static Node reverse(Node head) {

  if (head == null)
   return null;

  Node currentNode = head.next;
  Node previousNode = head;
  while (currentNode != null) {
   System.out.println("previous:" + previousNode +" current:"+currentNode);
   
   Node tempCurrent = currentNode.next;

   currentNode.next = previousNode;
   previousNode = currentNode;

   /* move to next node */
   currentNode = tempCurrent;
  }
  head.next = null;

  return previousNode;
 }

 static void print(Node head) {
  while (head != null) {
   System.out.print(head + " ");
   head = head.next;
  }
  System.out.println();

 }

 public static void main(String[] args) {
  
  Node head = new Node(1);
  head.next = new Node(2);
  head.next.next = new Node(3);
  head.next.next.next=new Node(4);
  
  /*print actual linked list*/
  print(head);

  Node newHead = reverse(head);

  /*print reversed linked list*/
  print(newHead);
 }
}

Output

Node[data=1] Node[data=2] Node[data=3] Node[data=4] 
previous:Node[data=1] current:Node[data=2]
previous:Node[data=2] current:Node[data=3]
previous:Node[data=3] current:Node[data=4]
Node[data=4] Node[data=3] Node[data=2] Node[data=1] 

Please share/ask more data structure problems with us. We will try to solve your problem in coming posts. Also share this post with your friends/classmates.

Principles of Testing - Software Testing Principles

Testing is a process to expose hidden defects. It is detecting errors and deviations from specification. It is verifying that a system satisfies its specified requirements or not. Here discuss principles of testing.

Seven General principles of Software testing

Principle 1. Exhaustive Input Testing
Exhaustive input testing means test all possible input condition as test cases. Test for all valid and invalid input conditions. But it is impossible to test a system for all input test cases.

Principle 2. Testing is creative and difficult
Second principle of testing is "testing is creative and difficult". Yes it requires creativity. It requires extensive domain knowledge. It also requires good testing methodology.

Principle 3. Prevention of defects
It is a procedure to fixing errors. If we find defects in early stages of development then its cost of fixing is less. Cost of fixing of any defect is too higher in later stages. So it's better to go with preventive approach.

Principle 4. Testing is risk based
Testing is risk based process. A risk is a loss associated with an event. Risk can be economical.
Suppose I try to test one module of system. I got some defects. I solved defects but due to modification some more defects appear in same module or other modules. So it is risky process. Sometime it increases the cost of testing.

Principle 5. Testing must be planned
Test planning is essential. Test Planning helps to solve many problems in system. Test plan covers points like requirement of testing, test priority, cost of testing, test team, test strategy, test tools etc. These factors effect on testing.

Principle 6. Testing requires independence
Testing must be unbiased. Unbiased testing is essential to objectively test quality of software. If testing is done by same developer who develop it then it may be biased. Developer can be an emotional attachment with its development. Developer who has to test his or her program parts will tend to be too optimistic. Chances of “blindness to their own errors”. So require to testing done by other than developer to make unbiased.

Principle 7. Provide expected results
It is important principle of testing. Test is done to check system is fulfilling user requirement or not. System must provide expected results. Testing checks that system pre defined specifications achieved or not.

 

String Reverse Example in Java Programming

It is a sample string reverse example. In this program read array of string from user and give output array of reverse string.
Here use StringBuffer class and create object name[] of StringBuffer class. Use method reverse() to reverse inserted string.

Program Code

//developed by Om Prakash
import java.util.*;
class StringReverse
{
public static void main(String argv[])
{
int size;
Scanner console=new Scanner(System.in);
System.out.println("*******STRING REVERSE*******");
System.out.print("How many string do you want to enter : ");
size=console.nextInt();
StringBuffer name[]=new StringBuffer[size+1];
String temp;
int i;
System.out.println("Enter "+size+" string : ");
for(i=0;i<=size;i++)
{
temp=console.nextLine();
name[i]=new StringBuffer(temp);
}
System.out.println("String after Reverse");
for(i=0;i<=size;i++)
{
name[i].reverse();
System.out.println(name[i]);
}
}
}

Output